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Abstract 

A  group  support  system  (GSS)  uses  a  combination  of  networked  personal 
computers,  software  that  collects,  manipulates,  and  aggregates  member’s  individual 
input,  and  human  facilitation  to  improve  the  group  decision-making  process.  Group 
support  systems  are  being  used  in  the  Air  Force  today  in  a  variety  of  capacities  and  in 
particular  by  the  Wamer-Robins  Air  Logistics  Center  (WR-ALC)  to  assess  acquisition 
risks.  GSS  facilitators  at  WR-ALC  are  interested  in  achieving  the  optimal  productivity 
out  of  their  GSS  system.  Prior  GSS  research  has  found  that  content  and  process 
anonymity  influence  problem  solving  groups.  However,  previous  studies  report  mixed 
results  on  which  levels  of  anonymity  positively  influence  group  performance. 

This  thesis  looked  at  content  and  process  anonymity  using  four  treatments  to 
explore  possible  explanations  for  the  mixed  results  found  in  previous  GSS  research.  The 
study  examined  numerous  theories  including  anonymity,  identification,  social  loafing, 
and  social  comparison.  An  experiment  was  developed  to  assess  how  content  and  process 
anon5miity  influence  participation  rates,  quality  of  the  group  decision,  consensus  levels, 
user  attitudes,  and  user  satisfaction  with  the  group  outcome.  Groups  participated  in 
conditions  of  total  anonymity,  process  anonymity,  and  no  anonymity.  The  no  anonymity 
condition  included  face-to-face  discussion  and  use  of  a  GSS  in  which  comments  were 
labeled  and  stations  were  labeled  with  placards.  The  results  of  the  study  were 
underwhelming.  In  general,  it  was  found  that  face-to-face  groups  participated  at  higher 
levels,  achieved  a  higher  group  decision  quality,  reached  higher  consensus  and  were  more 
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satisfied  with  the  group  outcome.  These  findings  cannot  be  attributed  to  the  lack  of 
process  and  content  anonymity  however,  because  the  GSS  labeled  with  placard  treatment 
did  not  achieve  the  same  results. 
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INFLUENCE  OF  ANONYMITY  IN  A 
GROUP  PROBLEM-SOLVING  ENVIRONMENT 
I.  Introduction 

A  Group  Support  System  (GSS)  is  the  combination  of  “communication, 
computer,  and  decision  technologies  to  support  problem  formulation  and  solution  in 
group  meetings  (DeSanctis  and  Gallupe,  1987:  589).”  The  basic  purpose  of  this 
technology  is  “to  increase  the  effectiveness  of  decision  groups  by  facilitating  the 
interactive  sharing  and  use  of  information  among  group  members  and  also  between  the 
group  and  the  computer  (Huber,  1984:  186).” 

A  GSS  will  normally  increase  participation  allowing  all  meeting  members  to 
participate  fully  without  regard  to  criticism,  since  idea  generation  can  be  anonymous 
(Jessup,  Connolly,  and  Galagher,  1990).  Consequently,  ideas  are  evaluated  on  their 
worth  rather  than  on  the  source  of  the  idea.  Anonymity,  however  also  allows  group 
participants  to  socially  loaf  (Sanna,  1992),  which  may  minimize  the  benefits  anonymity 
provides  to  other  group  members. 

Previous  research  has  shown  that  anonymity  can  influence  the  perceptions  and 
social  interactions  of  individual  group  members  (Pinsonneault  and  Heppel,  1997).  This 
influence  can  be  both  positive  (increased  participation)  or  negative  (social  loafing). 
Subsequently,  the  level  of  anonymity  provided  in  a  meeting  is  an  important  design 
decision  that  should  be  considered  when  setting  up  a  meeting.  A  GSS  provides  meeting 
managers  with  multiple  options  in  the  area  of  anonymity  that  should  be  considered.  This 
study  will  shed  light  on  these  options  and  highlight  the  strengths  and  weaknesses  of 
different  levels  of  anonymity. 
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1.1  Anonymity  Types 

Two  types  of  anonjonity  will  be  discussed  throughout  this  paper:  content 
anonymity  and  process  anonymity.  Valacich,  Jessup,  Dennis  and  Nunamaker  define 
content  anonymity  as  “the  extent  to  which  group  members  can  identify  the  source  of  a 
particular  contribution  by  recognizing  the  author  through  an  identifier  embedded  in  a 
contribution  (1992:  224).”  Knowing  the  author  does  not  mean  that  participation  can  be 
associated  with  a  specific  individual.  Embedded  identifiers  typically  use  code  names 
(e.g.,  Red,  Green,  Blue,  Yellow)  that  protect  the  individual  participant’s  identity. 
Protection  of  the  participant’s  identity  refers  to  process  anonymity.  For  the  purposes  of 
this  study,  content  anonymity  will  be  achieved  when  participants  do  not  have  comment 
labels.  When  comment  labeling  is  present,  participants  will  not  have  content  anonymity. 

Process  anonymity  is  defined  as  “the  extent  to  which  group  members  can 
determine  who  is  participating  by  directly  observing  who  is  making  a  contribution  to  the 
process  (Valacich  et  al.,  1992:  223).  Process  anonymity  is  achieved  when  participants 
cannot  directly  observe  (either  in  a  face-to-face  or  GSS  session)  when  another  participant 
is  contributing.  When  comment  labeling  is  present,  and  when  the  participants’  stations 
are  labeled  to  reflect  the  comment  labels  (via  placards),  participants  will  have  no  content 
or  process  anonymity.  Combining  content  and  process  anonymity  produces  three  levels 
of  anonymity  within  a  GSS:  process  and  content;  content  only;  and  no  anonymity. 
Investigating  the  relative  effects  of  these  three  levels  may  provide  insight  into  the 
equivocal  results  of  past  GSS  research. 
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1.2  The  Need  for  Meetings  and  GSS 

Corporate  America  spends  an  inordinate  amount  of  time  in  meetings-time  that  is 
often  poorly  utilized  and  unproductive.  Poole  and  DeSanctis  label  a  meeting  as  a  “cul  de 
sac  down  which  promising  ideas  are  lured  and  quietly  strangled  (1990, 173).”  Group 
Support  Systems  (GSS)  have  been  hailed  as  a  potential  solution  to  these  time-wasting 
events.  Research  suggests  that  groups  that  use  GSS  technology  can  often  be  more 
productive  and  satisfied  than  non-GSS  groups  (Dennis  and  Gallupe,  1993). 

Individual  managers  seldom  have  access  to  all  relevant  information,  so  when  truly 
important  decisions  have  to  be  made,  a  group  is  formed  to  make  the  decision  or  to  advise 
the  individual  who  must  make  it  (Hackman  and  Kaplan,  1974).  Group  discussion  enables 
participants  to  share  information  so  that  the  group  as  a  whole  can  access  a  larger  pool  of 
information  than  any  one  person  acting  alone  (Shaw,  1981).  The  exchange  of 
information  is  the  key  difference  between  individual  and  group  decision  making  —  and 
the  key  element  of  group  decision  making  (Dennis,  1996). 

As  demands  increase  to  “do  more  with  less”,  our  ability  to  improve  our  level  of 
productivity  through  the  effective  implementation  of  computers  will  continue  to  grow. 
From  the  1950s  through  the  1970s  productivity  gains  were  accomplished  through  the 
automation  of  repetitive  information  processing  tasks.  In  the  1980s,  the  proliferation  of 
computers  grew  at  a  rapid  pace  because  of  their  lower  costs,  increased  power  and 
decreasing  size.  Since  this  explosion  of  growth,  similar  advances  have  taken  place  in 
networking  technology.  The  interconnectivity  now  available  to  everyday  users  of  the 
computer  has  increased  their  usefulness  and  effectiveness.  Developing  ways  to  capitalize 
on  networked  computers  is  now  at  the  forefront  as  people  continue  to  look  for  ways  to 
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incorporate  technology  into  their  workday.  This  study  looks  at  how  anonymity  can  be 
adapted  into  group  problem-solving  processes  and  the  benefits  and  drawbacks  of  doing 
so. 


1.3  Groups  as  Barriers  and  Facilitators 

Properties  of  group  structure  and  social  interactions  act  both  as  barriers  and 
facilitators  to  the  effectiveness  of  decision-making  groups.  Barriers  may  include 
accountability,  information  processing,  production  blocking,  and  social  loafing. 
Facilitators  may  include  collective  learning  and  identifiability.  “Accountability,  the 
social  pressure  to  justify  one's  views  to  others,  has  repeatedly  been  shown  to  influence 
how  and  what  individuals  think”  (Green,  Visser  and  Tetlock,  2000: 1380).  Information 
processing  can  result  in  information  overload  if  the  group  is  too  large  or  if  the  task  is  not 
fully  understood.  This  can  hinder  the  group  communicative  process.  Production 
blocking  occurs  when  something  or  someone  prevents  verbalization  of  ideas  as  they 
occur  (Shepherd,  Briggs,  Reinig  and  Yen,  1995).  This  may  result  in  the  member 
becoming  too  distracted  to  contribute  new  ideas  or  result  in  social  loafing.  In  social 
loafing,  “individuals  tend  to  expend  less  effort  in  group  tasks  than  they  do  in  individual 
tasks,  unless  their  contribution  can  be  specifically  identified,  or  unless  they  believe  that 
their  contribution  is  critical  to  the  success  of  the  task  (Shepherd  et  al.,  1995:  524).” 

Hackman  (1995)  writes  that  fostering  collective  learning  can  result  in  a 
synergistic  gain  from  group  interaction.  Groups  that  share  what  is  learned  should  reach 
better  decisions.  Contrary  to  social  loafing,  some  group  members  want 
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acknowledgement  for  their  ideas.  They  want  to  be  identified  each  time  they  speak  and 
the  group  offers  them  a  forum  for  their  ideas  and  possibility  the  recognition  they  covet. 

GSS  technology  has  been  developed  to  enhance  the  positive  aspects  (facilitators) 
of  groups  while  minimizing  the  negative  factors  (barriers).  According  to  Hackman 
(1995),  “the  design  of  a  group — ^task  structure,  group  composition,  and  group  norms — 
should  promote  effective  task  behavior  and  lessen  the  chances  that  members  will 
encounter  built-in  obstacles  to  good  performance”  (411).  The  challenge  that  exists  today 
is  to  develop  ways  of  understanding,  designing  and  managing  groups  that  help  them  meet 
or  exceed  standards  of  team  effectiveness  (Hackman,  1995). 

A  Group  Support  System  may  help  teams  meet  and  exceed  these  standards.  The 
anonymity  a  GSS  can  provide  may  reduce  the  importance  of  team  member’s 
demographic  attributes  (e.g.,  gender,  age,  or  rank)  or  behavioral  style  (e.g.,  talkativeness). 
In  addition,  member  behavior  can  be  self-regulated  through  the  structure  available  in  a 
GSS.  The  challenge  is  to  create  a  task  environment  that  promotes  a  shared  commitment 
among  members  and  minimizes  social  loafing. 

By  allowing  anonymous  communications,  GSS  are  expected  to  reduce  inhibition 
and  evaluation  apprehension,  enhance  participation,  increase  the  number  of  ideas 
generated,  and  improve  the  quality  of  decisions  (Nunamaker,  Dennis,  Valacich,  Vogel, 
and  George,  1993).  Some  GSS  studies  have  found  strongly  positive  results  when 
comparing  GSS  meetings  to  face-to-face  meetings,  while  other  studies  found  mildly 
positive  results,  and  others  found  mixed,  neutral,  or  negative  results  (Lam,  1997). 

Research  on  Group  Support  Systems  has  primarily  focused  on  the  efficiency 
realized  by  the  technology,  the  abilities  of  facilitators,  and  the  results  of  GSS  meetings 
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when  compared  to  face-to-face  groups  (Nunamaker,  Briggs,  Mittleman,  Vogel,  and 
Balthazard  1997:  202,  Anson,  Bostrom,  and  Wynne,  1996).  Each  of  these  previous 
efforts  called  for  further  investigation  of  other  contextual  factors  that  may  influence  GSS 
processes  and  outcomes.  The  goal  of  this  line  of  research  is  to  discover  meeting 
processes  and  designs  that  can  consistently  increase  the  quantity  and  quality  of  comments 
and  ideas  generated  in  a  meeting,  and  further  the  likelihood  of  improving  decision  quality 
(George  and  others,  1990).  The  future  use  of  GSS  will  be  determined  by  the  potential  to 
reach  quality  decisions  in  a  GSS  environment.  The  importance  or  lack  of  importance  of 
anonymity  will  be  realized  by  determining  the  impact  anonymity  has  on  those  decisions. 
Pinsonneault  and  Heppel  (1997)  believe  that  more  laboratory  and  field  experiments 
should  be  conducted  to  determine  the  impact  of  anonymity  on  group  processes.  Given 
the  conflicting  findings  on  GSS  effectiveness  when  comparing  face-to-face  and  GSS 
meetings,  and  the  promising  results  of  recent  research  on  contextual  factors  of  GSS 
design,  this  topic  is  worthy  of  further  investigation. 

1.4  Research  Applicability  to  the  United  States  Air  Force 

Since  the  late  1980s,  the  United  States  Air  Force  has  faced  reductions  in 
manpower  and  budgets.  At  the  same  time,  demands  have  increased  for  more  flexible, 
cost-effective  operations.  The  days  of  attending  expensive  and  time-consuming  meetings 
at  destinations  around  the  world  are  rapidly  dwindling.  This  has  resulted  in  the  demand 
to  “do  more  with  less”  and  to  find  more  efficient  and  effective  ways  to  hold  meetings. 

Reduced  business-cycle  times  and  improved  process  efficiencies  are  becoming 
increasingly  critical  to  Air  Force  effectiveness  (Air  Force  Research  Laboratory 
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Sustainment  Logistics  Branch,  1999).  The  implementation  of  Lean  Logistics  during  this 
timeframe  was  in  response  to  the  Air  Force’s  quest  to  streamline  its  logistics 
infrastructures  and  processes  (Office  of  the  Secretary  of  Defense,  1996). 

The  Lean  Logistics  program  focuses  on  improving  operational  units'  capabilities 
by  applying  modem  business  practices  across  all  logistics  functions  and  processes.  The 
program  suggests  the  use  of  just-in-time  asset  management  and  repair  systems  instead  of 
the  traditional  demand-driven,  just-in-case  asset  management  systems.  The  Air  Force  is 
advancing  innovative  concepts,  such  as  the  use  of  information  systems,  to  support  Lean 
Logistics  in  providing  effective  and  less  costly  depot-level  maintenance  operations 
(Office  of  the  Secretary  of  Defense,  1996,  Office  of  the  Secretary  of  the  Air  Force,  1997). 

With  DoD’s  focused  logistics  and  the  Air  Force’s  agile  combat  support  initiatives, 
the  logistics  community  must  continually  re-evaluate  the  processes  used  to  support  the 
warfighter.  The  emphasis  on  cost  cutting  has  also  placed  considerable  pressure  on 
logistics  units  at  the  base  and  depot  levels  to  streamline  their  operations.  Depots  are  also 
facing  increased  competition  from  commercial  firms  for  aircraft  repair  and  maintenance 
business.  As  a  result.  Air  Force  logistics  units  are  confronted  with  the  need  to  change 
their  processes  in  an  attempt  to  incorporate  Lean  Logistics  to  ensure  success  in  this 
increasingly  competitive  and  dynamic  environment.  Implementation  of  Lean  Logistics 
concepts  will  require  increased  communication  and  collaboration  among  the  affected 
units  (Air  Force  Research  Laboratory  Sustainment  Logistics  Branch,  1999,  Air  Force 
Research  Laboratory  Sustainment  Logistics  Branch,  1998). 

In  support  of  Lean  Logistics,  the  Air  Force  Research  Laboratory  (AFRL)  has 
developed  a  distributed  computer-mediated  decision  support  system  to  allow 
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maintenance  depots,  base  logistics  units,  and  command  headquarters  to  accomplish 
process  redesign.  The  system  is  composed  of  two  programs.  The  first  is  called  RAPTR, 
or  Readiness  Assessment  and  Planning  Tool  Research.  RAPTR,  not  in  itself  a  distributed 
GSS,  provides  a  business  process  reengineering  toolkit  intended  to  assist  logisticians  and 
managers  in  implementing  changes  within  their  organizations.  The  program  will  identify 
processes  for  reengineering  and  offer  remedies  to  address  them.  (Air  Force  Research 
Laboratory  Sustainment  Logistics  Branch,  1998). 

The  second  component  of  the  proposed  system  is  the  Depot  Operations  Modeling 
Environment,  (DOME).  It  is  this  component  that  uses  distributed  GSS  technology.  The 
goal  of  the  DOME  system  is  to  aid  in  the  design  and  modeling  of  Air  Force  logistics 
processes  using  a  collaborative  environment  which  establishes  connectivity  between 
dispersed  groups  and  installations.  The  DOME  system  relies  heavily  on  an  existing 
commercial  product  known  as  GroupSystems®  developed  by  the  University  of  Arizona. 
Many  of  the  tools  developed  for  the  DOME  project  extend  the  capabilities  of 
GroupSystems®  to  offer  greater  functionality.  The  DOME  system  has  been  successfully 
installed  and  demonstrated  at  the  Wamer-Robins  ALC,  Robins  AFB,  Georgia  and  the 
366th  Wing  at  Mountain  Home  AFB,  Idaho.  (Final  Dome,  Air  Force  Research 
Laboratory  Sustainment  Logistics  Branch,  1998) 

These  examples  demonstrate  the  fact  that  GSS  use  is  becoming  more  widespread 
throughout  the  Air  Force  to  meet  the  tougher  demands  of  a  “do  more  with  less”  military 
environment.  Determining  what  level  of  anonymity  can  optimize  the  use  of  Group 
Support  Systems  is  essential  in  facilitating  their  use  and  ultimately  their  success. 


8 


1.5  Problem  Statement  and  Purpose  of  Research 

This  study  will  be  one  of  four  theses  that  delve  into  different  aspects  of  GSS 
research.  All  four  studies  will  use  the  same  experimental  sessions  to  collect  data,  but 
each  study  will  concentrate  on  different  manipulations,  measured  independent  variables, 
and  outcomes.  One  study  will  evaluate  the  influence  of  the  participant’s  individual 
characteristics  in  a  GSS  environment  (Hartmann,  2001).  A  second  study  will  look  at  the 
impact  an  expert  (or  perceived  expert)  has  on  a  problem-solving  exercise  in  a  GSS 
environment  (Thompson,  2001).  The  third  will  examine  how  real  time  feedback  through 
comment  labeling  and  process  feedback  will  influence  the  quantity  and  quality  of  ideas 
generated  in  a  GSS  environment  (Denney,  2001).  The  purpose  of  this  study  was  to 
determine  the  influence  of  varying  levels  of  anonymity  on  the  group  problem-solving 
process.  Success  will  be  determined  by  evaluating  group  members’  satisfaction  levels 
with  both  the  process  and  the  group  outcome,  group  participation  levels,  group 
consensus,  and  ultimately  group  decision  quality.  For  the  purposes  of  this  study, 
anonymity  will  be  defined  as  “the  identifiability  of  group  member  contributions  (Jessup 
and  Connolly,  1990:  314).” 

Many  studies  have  looked  into  the  impact  of  anonymity  in  a  GSS  environment  by 
comparing  these  meetings  with  face-to-face  meetings.  However,  Connolly,  Jessup  and 
Valacich  (1990)  believe  that  based  on  the  potential  power  of  the  anonymity  manipulation 
in  a  GSS  that  there  seems  to  be  real  research  potential  in  further  clarifying  the  interaction 
between  task,  group  and  individual  characteristics  and  anonymity.  This  study  will  look 
closer  at  the  different  levels  of  anonymity  a  GSS  can  provide  and  compare  them  with 
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each  other  and  with  face-to-face  meetings  in  an  effort  to  identify  the  best  setting  for 
achieving  highly  productive  meetings  and  satisfied  participants. 

1.6  Summary 

A  GSS  offers  problem-solving  teams  a  variety  of  system  configurations  and 
provides  today’s  teams  with  many  possible  levels  of  anonymity.  Subsequently, 
identifying  the  level  of  anonymity  which  results  in  the  highest  level  of  consensus, 
increased  participation  levels,  or  the  highest  quality  decision  would  be  very  valuable  to 
leaders  who  are  establishing  problem-solving  teams. 

While  many  studies  have  been  published  on  the  influence  of  anonymity,  little 
research  has  been  conducted  on  different  levels  of  anonymity  in  problem-solving  groups 
outside  of  anonymous  GSS  and  face-to-face.  This  study  will  identify  which  level  of 
anonymity  will  enhance  the  quality  of  a  group’s  performance,  increase  their  level  of 
consensus,  facilitate  participation  and  increase  user  satisfaction  with  the  process  and  the 
group  outcome. 

1.7  Sequence  of  Presentation 

Chapter  n  reviews  the  relevant  GSS  literature  with  emphasis  on  literature  that 
pertains  to  the  dependant  variables  studied  in  this  thesis.  Chapter  HI  describes  the 
methodology  used  to  conduct  the  research  for  this  study.  Chapter  IV  presents  the  data 
collected  and  results  of  this  study.  Finally,  Chapter  V  interprets  the  results  obtained  in 
Chapter  IV  along  with  implications,  study  limitations,  and  suggestions  for  future 
research. 


10 


n.  Literature  Review 


2.1  Introduction 

This  chapter  explores  group  decision-making  approaches  and  the  evolution  of 
GSS  research  from  the  initial  studies  conducted  in  the  1970s  to  current  research  that 
focuses  on  the  technology’s  ability  to  support  decision-making  groups.  Emphasis  will  be 
placed  on  the  role  of  anonymity  and  comment  labeling  in  problem-solving  groups,  and 
subsequently  their  impact  on  participation,  group  performance,  consensus,  and 
satisfaction  with  the  process  and  the  outcome.  Finally,  this  chapter  presents  the 
theoretical  basis  and  the  hypotheses  investigated  for  the  research  contained  in  this  study. 

2.2  GSS  Development  and  Process 

The  forerunner  of  Group  Support  Systems  (GSS)  was  developed  at  the  University 
of  Texas  when  Dr.  Gerald  R.  Wagner  created  a  system  called  Mindsight.  Mindsight 
provided  the  inspiration  for  the  basic  configuration  for  GSS  replete  with  computer 
terminals,  projection  screens  and  decor  designed  to  promote  group  discussions.  The 
lineage  of  the  GSS  system  can  be  traced  to  the  University  of  Arizona  in  the  early  1980s. 
There  they  developed  software  called  Plexsys  and  the  first  GSS  room,  which  was  called 
the  PlexCenter.  Both  were  developed  to  support  group-process  talks  such  as  idea 
generation,  idea  organization,  voting,  and  policy  formulation  (Wagner,  Wynne,  and 
Mennecke,  1993). 

The  system  used  in  this  study,  Group  Systems,  has  resulted  in  numerous  research 
studies  over  the  years.  With  Group  Systems,  a  facilitator  runs  the  session,  users  are 
placed  in  front  of  terminals  (either  locally  or  distributed),  and  a  toolkit  is  provided  for 
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each  user.  Tools  from  the  toolkit  deemed  pertinent  to  the  session  are  selected  by  the 
facilitator  and  made  available  to  the  users.  The  session  follows  the  outline  presented  in 
the  Group  Systems  Agenda.  The  Agenda  items  each  are  linked  to  a  specific  tool  from  the 
tool  kit,  which  help  keep  the  meeting  structured.  For  instance,  a  brainstorming  agenda 
item  would  use  the  Brainstorming  tool  and  the  need  to  vote  would  use  the  Group  Systems 
Vote  tool.  As  the  agenda  progresses,  the  facilitator  can  incorporate  new  tools  from  the 
toolkit  as  needed.  During  the  meeting,  the  GSS  display  can  be  displayed  on  a  large 
screen.  Often  this  screen  is  referred  for  clarification  or  further  discussion.  The  GSS  can 
provide  a  group  memory  by  recording  all  electronic  comments.  This  allows  participants 
to  “decouple  themselves  from  the  group  to  pause,  think,  type  comments,  and  rejoin”  the 
session  without  missing  anything  (Nunamaker,  Dennis,  Valacich,  Vogel  and  George, 
1993:  132).  Once  the  meeting  concludes,  the  system  maintains  a  complete  record  of  the 
session  for  future  reference. 

2.3  Group  Research  Background 

The  conceptual  framework  for  the  study  of  groups  starts  with  two  givens:  the 
characteristics  of  individual  members  of  the  group  and  the  environment  in  which  those 
people  are  immersed  (McGrath,  1984).  Individual  characteristics  include  demographics 
such  as  gender,  age,  rank,  and  level  of  education.  They  also  include  personality,  beliefs, 
attitudes  and  values,  moods,  state  of  mind,  needs,  motives,  goals,  and  expectations 
(McGrath,  1984).  Environmental  characteristics  include  the  cultural  and  technological 
environment.  The  cultural  environment  includes  socially  transmitted  behavior  patterns 
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based  on  beliefs  and  institutions.  The  technological  environment  includes  the 
characteristics  of  the  system  being  used. 

A  conceptual  framework  for  the  study  of  anonymity  was  developed  by  Valacich 
et  al.  (1992).  Their  model  described  five  factors  that  influenced  anonymity.  The  factors 
map  very  well  to  McGrath’s  conceptual  framework  (see  Table  1). 

Table  1. 

Linkage  between  McGrath  and  Valacich,  Jessup,  Dennis  and  Nunamaker. 

McGrath  Valacich,  Jessup,  Dennis  and  Nunamaker 

Individual  Characteristics  Group  Composition 

Cultural  Environment  Group  History,  Group  Size  and  Member  Proximity 

Technological  Environment  System  Characteristics 


The  focal  point  of  Valacich  et  al’s  (1992)  model  are  the  two  types  of  anonymity. 
Content  anonymity  is  provided  when  embedded  identifiers,  which  identify  a 
contribution’s  source,  are  absent.  Process  anonymity  is  provided  when  the  contributor 
cannot  be  determined  by  direct  observation.  A  GSS  normally  provides  a  greater  amount 
of  process  anonymity  over  face-to-face  groups  when  GSS  participants  and  comments  are 
unlabeled.  Increased  process  anonymity  is  also  more  prevalent  in  a  GSS  because 
contributions  cannot  be  directly  attributed  to  participants  through  direct  observation. 
Process  anonymity,  however,  can  be  minimized  in  a  GSS  environment  if  comments  are 
labeled  and  participants  are  identified  through  placards  linking  the  labels  to  the 
participants. 
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The  individual’s  inclination  to  act,  and  the  nature  of  the  act,  is  influenced  by  the 
motives  and  actions  of  other  group  members.  The  model  introduced  by  Valacich  et  al. 
(1992)  separates  effects  on  the  individual  because  the  group  process  is  a  compilation  of 
the  individual  communicative  episodes.  Some  of  the  individual  effects  are  social  loafing, 
social  comparison,  and  production  blocking.  The  effect  on  the  individual  can  be 
evaluated  by  measuring  their  satisfaction  with  the  process,  their  satisfaction  with  the 
outcome,  and  their  participation  rate.  The  model  further  breaks  down  the  effects  of 
anonymity  on  the  group  into  effects  on  the  group  process  and  effects  on  the  group 
outcome.  These  can  be  measured  by  evaluating  the  quality  of  the  decision  and  the  level 
of  consensus  reached  by  the  group.  This  study  modifies  the  model  developed  by 
Valacich  et  al.  (1992),  incorporating  McGrath’s  conceptual  framework  of  groups.  (See 
Figure  1). 


Factors  Influencing 
Anonymity 

Technological 
Environment  ^ 

Cultural 

Environment 

Individual  _ ^ 

Characteristics 


Types  of 


Effects  on  the 
Individual 

Receiver 


Sender 


>• 


Effects  on  the 
Group 


Group 

Process 


Group 

Outcome 


Figure  1.  Conceptual  framework  for  GSS  anonymity  adapted  from  Valacich,  Joseph  S., 
Leonard  M.  Jessup,  Alan  R.  Dennis,  and  J.F.  Nunamaker  Jr.  “A  Conceptual  Framework 
of  Anonymity  in  Group  Support  Systems.”  Group  Decision  and  Negotiation.  1:  223 
(1992). 
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When  studying  groups,  the  environment  can  take  an  infinite  number  of  forms 
because  of  the  many  differences  between  people  (individual  characteristics)  and  the 
different  cultural  and  technological  situations  in  which  the  groups  can  be  placed.  This 
makes  it  important  to  carefully  select  the  environment  and  the  participants  to  be  included 
in  a  study. 

Some  common  problems  experienced  by  decision-making  groups  include  the 
extreme  influence  exerted  by  high-status  members,  the  lack  of  acknowledgement  of  low- 
status  members’  ideas,  and  a  low  tolerance  exhibited  toward  minority  or  controversial 
opinions  (DeSanctis  and  Gallupe,  1987).  Group  members  with  low  status  characteristics 
(age,  rank  and  level  of  education)  have  difficulty  achieving  influence  over  group 
decisions  (Ridgeway,  1982).  DeSanctis  and  Gallupe  (1987)  add  that  some  group 
members  are  reluctant  to  participate  because  of  their  shyness,  low  status  or  the 
controversial  ideas  being  discussed.  Another  significant  problem  that  arises  in  problem 
solving  groups  is  production  blocking.  “Production  blocking  occurs  when  something 
prevents  verbalization  of  ideas  as  they  occur.  One  may  forget  an  idea  while  waiting  for  a 
turn  to  speak,  or  may  devote  attention  to  remembering  an  idea,  becoming  too  distracted  to 
generate  new  ideas  (Shepherd,  Briggs,  Reinig,  Yen,  and  Nunamaker,  1995:  157).”  With 
the  GSS,  all  participants  can  contribute  simultaneously  and  all  ideas  are  immediately 
displayed  for  all  group  members  to  review.  “Because  nobody  has  to  wait  for  a  turn  to 
speak,  production  blocking  is  eliminated  (Shepherd  et  al.,  1995:  157).”  This  study  aims 
to  answer  many  of  the  questions  revolving  around  the  applicability  of  content  and  process 
anonymity  in  trying  to  overcome  these  common  problems. 
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2.4  Evolution  of  GSS  Research 

The  beginnings  of  GSS  empirical  research  can  be  traced  back  to  a  study 
conducted  by  Chapanis  and  his  associates.  He  was  among  the  first  researchers  to  study 
computer  messaging  versus  face-to-face  communication.  In  his  study  (Chapanis,  1972) 
two-person  teams  solved  problems  using  either  computer  messaging,  remote  handwriting, 
audio  only,  or  face-to-face  discussions.  He  found  that  groups  using  computer  messaging 
took  longer  to  solve  the  problems  and  sent  fewer  messages  than  face-to-face  groups. 

The  early  1980’s  saw  the  emergence  of  exploratory  studies  of  computer-based 
support  for  group  processes.  “These  early  efforts  were  conducted  by  pioneers  who 
recognized  that  computer  and  communication  technology  offered  the  potential  to  improve 
the  productivity  of  groups  (Jessup  and  Valacich,  1993:  62).”  These  early  studies 
attempted  to  develop  GSS  systems  and  examine  their  affects.  The  lessons  learned  during 
this  early  phase  of  research  included  (1)  GSS  has  the  potential  to  improve  group 
processes  and  performance,  (2)  better  GSS  systems  are  needed,  and  (3)  in  order  to 
understand  their  true  impact,  more  rigorous  GSS  research  is  needed  (Dennis  and  Gallupe, 
1993). 

The  next  phase  of  GSS  research  primarily  consisted  of  studies  that  compared 
face-to-face  groups  with  computer  mediated  groups.  These  studies  often  evaluated 
decision  quality,  time  to  reach  decision,  consensus  and  participant  satisfaction  with  both 
the  process  and  the  group  outcome.  “The  results  of  these  early  experiments  can  be 
summed  up  in  one  word:  mixed  (Dennis  and  Gallupe,  1993:  66).”  What  these  studies 
did  make  clear  was  that  using  GSS  made  a  difference.  Further  efforts  are  needed  to 
determine  what  factors  cause  the  mixed  findings. 
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At  this  point,  the  majority  of  studies  were  conducted  in  the  field.  One  of  the  areas 
focused  on  during  this  time  was  anonymity.  Again,  the  results  were  mixed.  In  general, 
GSS  field  studies  found  positive  reactions  to  GSS  and  anonymity  (Dennis  and  Gallupe, 
1993).  However,  Jessup  and  Connolly  (1990)  stated  that  “we  are  far  from  understanding 
the  true  impacts  of  anonymous  GSS  interaction  (314).” 

More  recent  research  has  shifted  away  from  investigation  of  the  capabilities  of  the 
technology  itself  toward  investigating  the  group  dynamics  of  GSS  supported  group 
meetings.  Many  of  these  studies  have  looked  closely  at  the  number  of  comments 
contributed  and  the  anonymity  a  GSS  can  provide  to  meeting  members. 

2.5  Role  of  Anonymity  in  Problem-Solving  Groups 

The  anonymity  provided  through  a  GSS  session  has  been  hailed  as  the  primary 
way  through  which  a  GSS  helps  problem-solving  groups  overcome  process  losses.  Kerr 
and  Bruun  (1981)  define  process  loss  as  “the  difference  between  the  group’s  potential 
and  actual  performance  (224).”  By  keeping  the  identity  of  GSS  participants  unknown, 
anonymity  reduces  the  fear  of  social  disapproval  and  of  evaluation,  increases  the  number 
of  ideas  generated,  and  lowers  inhibition  and  censorship  (Pinsonneault  and  Heppel, 
1997).  Research  has  also  shown  that  keeping  the  identity  of  members  involved  in  the 
GSS  unknown  can  bring  negative  results  to  the  meeting  (DeSanctis  and  Gallupe,  1987; 
Jessup,  1989;  Jessup,  Connolly,  and  Tansik,  1990;  Nunamaker  et  al.,  1988). 
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2.5.1  Problem-solving  Methods 

There  are  many  ways  in  which  groups  can  strive  to  reach  a  decision.  They 
include  the  “Plop”  method,  decision  by  authority  rale,  decision  by  minority  rule  and 
decision  by  majority  rale.  The  following  paragraphs  briefly  explain  each  of  these 
methods  and  how  different  levels  of  anonymity  can  be  applied  to  each. 

Schwartz  (1994)  defines  the  “Plop”  method  as  “someone  suggests  an  idea  and, 
before  anyone  else  has  said  anything  about  it,  someone  else  suggests  another  idea,  until 
the  group  eventually  finds  one  it  will  act  on”  (60).  This  is  probably  the  most  common 
problem-solving  method  implemented  by  groups  (Schwartz,  1994).  The  problem  with 
this  approach  is  ideas  are  rejected  before  they  are  considered.  “But  because  the  rejections 
have  been  simply  a  common  decision  not  to  support  the  idea,  the  proposers  feel  that  their 
suggestions  have  been  ‘plopped’  (Schwartz,  1994:  60).”  This  problem-solving  method  is 
likely  to  occur  when  the  group  has  a  dominant  person  or  when  there  are  varying  levels  of 
status  within  the  group.  The  “Plop”  method  is  less  likely  in  face-to-face  groups  with 
equal  status.  One  of  the  benefits  of  a  GSS,  the  elimination  of  production  blocking,  can 
help  reduce  the  likelihood  of  “plopping.”  Participants  can  contribute  equally  because  all 
participants  have  to  type  their  inputs.  This  reduces  the  impact  of  “plopping.” 
Subsequently,  the  less  process  anonymity  you  have  the  greater  the  likelihood  of 
“plopping.”  However,  content  anonymity  should  have  little,  if  any  effect,  on  “plopping.” 
With  high  content  anonymity,  ideas  are  either  addressed  or  ignored  one  at  a  time. 

In  the  decision  by  authority  method,  groups  use  a  power  structure  that  has  one 
person  with  ultimate  authority  over  the  final  decision.  The  problem-solving  group  can 
discuss  the  problem  at  length  and  make  recommendations,  but  the  anointed  leader  has  the 
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final  input.  “Whether  this  method  is  effective  depends  a  great  deal  upon  whether  the 
chairman  is  a  sufficiently  good  listener  to  have  culled  the  right  information  on  which  to 
make  the  decision  (Schwartz,  1994:  60).”  It  would  be  difficult  (if  not  impossible)  for  a 
known  power  structure  to  evolve  if  content  and  process  anonymity  are  provided. 
However,  if  only  content  anonymity  is  provided  it  is  possible  for  some  type  of  power 
structure  to  evolve.  As  inputs  are  made,  a  trend  could  develop  in  which  a  certain  label(s) 
(e.g.,  Green,  Blue)  is  thought  to  be  the  authority  on  the  topic.  This  could  result  in 
decision  by  authority  although  comments  cannot  be  associated  directly  with  the 
individuals  participating. 

“One  of  the  most  often  heard  complaints  of  group  members  is  that  they  feel 
railroaded  into  some  decision  (Schwartz,  1994:  60).”  Decision  by  minority  rule  often 
occurs  when  small  subsections  of  the  group  use  tactics  that  produce  action  and  ultimately 
decisions,  but  that  occur  without  the  consent  of  the  majority.  This  often  occms  when  a 
group  of  high-ranking  members  unite  and  take  action  before  the  majority  has  a  chance  to 
respond.  This  method  often  is  successful  for  the  minority  because  the  majority  remains 
silent  and  “silence  means  consent  (Schwartz,  1994:  61).”  The  use  of  this  problem¬ 
solving  method  can  be  reduced  when  content  and  process  anonymity  are  provided. 

Decision  by  majority  rule  is  one  of  the  most  well  known  methods  of  problem¬ 
solving.  Two  simple  implementations  of  this  method  include  polling  everyone’s  opinion 
following  a  period  of  discussion  or  asking  for  a  formal  vote  after  the  discussion  period. 
Often  decisions  made  by  this  method  are  not  well  implemented,  even  by  the  group  that 
made  the  decision  (Schwartz,  1994).  The  primary  problem  behind  voting  is  that  it  creates 
coalitions  and  the  minority  feels  misunderstood  and  often  resentful  (Schwartz,  1994). 
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Group  Support  Systems  attempt  to  minimize  the  problems  associated  with  each  of 
these  methods  and  ensure  that  each  participant’s  voice  is  “heard”  during  the  group 
discussion  period.  More  control  can  be  exerted  through  the  technology  to  minimize  the 
impact  of  a  dominant  group  subset  or  one  dominant  person  because  each  participant’s 
conunents  remain  on  the  screen  throughout  the  session.  “Problems  of  group  think^ 
pressures  to  conformity,  and  dominance  of  the  group  by  strong  personalities  or 
particularly  forceful  speakers  are  minimized  within  a  GSS  environment  (Nunamaker, 
Applegate  and  Konsynski,  1988:  839).” 

2.5.2  Social  Loafing 

One  way  content  and  process  anonymity  can  decrease  participation  is  through 
social  loafing.  Latane,  Williams  and  Harkins  (1979)  define  Social  Loafing  as  “a  decrease 
in  individual  effort  due  to  the  social  presence  of  other  persons  (823).”  Literature  on 
social  loafing  suggests  that  accountability  is  one  of  the  essential  factors  in  ensuring  that 
each  team  member  contributes  to  the  best  of  his/her  ability  (Hollenbeck,  1998).  Users 
may  participate  less  because  they  feel  that  anonymity  will  protect  them  from  group 
pressures  to  perform.  If  participants  lack  process  and  content  anonymity  they  are 
accountable  (either  through  embedded  identifiers  or  through  direct  observation)  for  their 
participation  or  lack  thereof. 

Williams  et  al.  (1981)  have  shown  that  when  participants  were  led  to  believe  that 
their  outputs  could  be  individually  identified,  social  loafing  was  eliminated.  However, 
Harkins  and  Jackson  (1985)  have  shown  that  the  identifiability  of  individual  performance 
alone  is  insufficient  to  eliminate  social  loafing.  In  the  Harkins  and  Jackson  (1985)  study. 


20 


social  loafing  was  eliminated  only  when  outputs  were  identifiable  (content  anonymity) 
and  when  participants  felt  they  were  performing  the  same  task.  In  this  instance,  they  felt 
they  could  be  compared  with  other  co-workers. 

2.5.3  Social  Comparison 

Social  comparison  is  a  phenomenon  wherein  people  match  their  rate  of 
performance  to  the  rate  of  the  people  working  around  them.  Participants  working  in  an 
environment  where  others  are  performing  at  a  high  level  also  tend  to  perform  highly. 
Participants  working  in  an  environment  where  others  are  performing  at  a  low  level  match 
the  inferior  performance  rate  (Goethals  and  Darley,  1987).  One  study  found  that  the 
productivity  of  manual  brainstorming  sessions  can  be  improved  by  inducing  group 
members  to  compare  themselves  to  a  mythical  average  group  (Paulus,  Dzindolet,  Poletes, 
and  Camacho,  1993).  Content  and  process  anonymity  prevents  this  comparison. 

The  level  of  anonymity  (both  content  and  process)  provided  in  a  GSS 
environment  can  thus  prove  to  be  detrimental  to  participants  because  they  have  no  means 
of  comparing  their  performance  to  others.  This  can  be  overcome  by  comparing 
participant’s  performance  with  an  average  group  performance  level  derived  from 
previous  sessions.  Shepherd  et  al.  (1995)  found  social  loafing  may  actually  occur  in 
anonymous  electronic  brainstorming  sessions  (EBS),  and  that  invoking  social  comparison 
with  a  mythical  average  group  reduces  the  social  loafing  phenomena.  They  added  that 
any  effort  to  compare  them  with  an  average  group  encourages  them  to  work  “quickly  at 
the  beginning  of  the  session  and  will  lead  to  high  productivity  throughout  the  session 
(Shepherd  et  al.,  1995:  530).” 
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2.6  Hypotheses 

The  body  of  empirical  GSS  research  has  grown  rapidly  since  the  early  1980s. 
This  section  organizes  these  studies  into  the  conceptual  framework  outlined  earlier  in  this 
chapter  and  develops  hypothesis  relevant  to  each  of  these  constructs.  A  revised 
conceptual  framework  is  depicted  in  Figure  2.  The  constructs  used  in  each  of  the 
hypothesis  are  included  under  effects  on  the  individual  and  effects  on  the  group.  Effects 
on  the  individual  are  analyzed  by  measuring  the  level  of  user  participation,  user  attitudes 
and  user  satisfaction  with  the  group  outcome.  Decision  Quality  and  Group  Consensus  are 
evaluated  to  determine  the  effect  of  anonymity  on  the  group. 

Factors  Influencing 

>■ 


Anonymity 

Technological 

Environment 

Cultural 

Environment 

Individual 

Characteristics 


Types  of 


Effects  on  the 
Individual 


Receiver  /  Sender 

HI :  User  Participation 

H4:  User  Attitudes 

H5;  User  Satisfaction  with 
Group  Outcome 


Effects  on  the 
Group 

Group 

Process/Outcome 

H2:  Decision  Quality 
H3:  Group  Consensus 


Figure  2.  Conceptual  framework  for  GSS  anonymity  with  study  constructs  incorporated 
adapted  from  Valacich,  Joseph  S.,  Leonard  M.  Jessup,  Alan  R.  Dennis,  and  J.F. 
Nunamaker  Jr.  “A  Conceptual  Framework  of  Anonymity  in  Group  Support  Systems.” 
Group  Decision  and  Negotiation.  L  223  (1992). 


Previous  research  has  looked  at  process  anonymity  by  comparing  GSS  use  to 
manual  group  meetings.  Content  anonymity  has  been  studied  by  comparing  labeled  GSS 
to  unlabeled  GSS  meetings.  Both  types  of  studies  have  tended  to  look  at  the  same 
dependent  variables.  Numerous  studies  and  their  findings  are  listed  chronologically  in 
Table  2  and  3.  Table  2  lists  studies  that  compared  results  in  GSS  sessions  with  face-to- 
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face  groups.  The  studies  included  looked  at  a  variety  of  computerized  conferencing 
methods  including  Electronic  Brainstorming  (EBS),  computer-aided  conferencing  and 
typical  GSS.  The  studies  listed  in  Table  2  were  included  because  they  analyze  the  effects 
of  process  anonymity.  The  face-to-face  groups  have  no  process  anonymity  and  GSS 
groups,  if  configured  with  labels  and  placards,  have  process  anonymity.  Two  significant 
patterns  emerge  from  these  studies:  GSS  groups  reach  a  higher  quality  decision  and  face- 
to-face  groups  achieve  higher  levels  of  consensus.  Another,  less  significant,  pattern  was 
found  in  the  area  of  participation  rates.  Many  of  the  studies  evaluated  equality  of 
participation  (instead  of  number  of  comments)  and  found  that  GSS  participants  more 
equally  participated.  These  studies  patterns  lead  into  the  hypothesis  discussed  in  the 
following  sections.  Table  3  lists  studies  that  compared  results  in  unlaheled  GSS  sessions 
with  labeled  GSS  sessions.  These  studies  primarily  address  the  effects  of  content 
anonymity.  The  unlabeled  GSS  sessions  have  both  content  and  process  anonymity  and 
the  labeled  GSS  sessions  only  have  process  anonymity.  Many  of  the  content  anonymity 
studies  only  evaluated  participation  rate,  which  made  finding  patterns  for  the  other 
constructs  difficult.  One  clear  pattern  was  that  groups  with  content  anonymity  would 
achieve  higher  participation  rates.  The  findings  listed  in  table  3  are  discussed  in  the 
following  sections  and  in  this  study’s  hypothesis. 

2.6.1  Influence  of  Anonymity  on  User  Participation 

Some  people  talk  more  than  others  do,  while  others  tend  to  be  more  reserved  with 
their  comments.  In  addition,  “persons  who  talk  more  get  talked  to  more  (McGrath, 
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Table  2.  (continued) 
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1984:145).”  The  person  who  talks  the  most  tends  to  direct  their  comments  to  the  entire 
group,  while  most  other  group  members  tend  to  direct  their  comments  to  specific  group 
members,  with  the  most  talkative  member  being  the  direction  of  most  cormnents 
(McGrath,  1984). 

In  addition,  “any  given  individual’s  rate  of  interaction  will  be  affected  by  the 
individual’s  ‘position’  in  the  group  (McGrath,  1984:146).”  This  position  is  determined 
by  their  location  at  the  meeting  table,  by  their  status  position  in  the  group,  by  their 
motivation  level,  and  by  their  perceived  value  to  the  group  (McGrath,  1984). 

Finally,  nonverbal  aspects  play  a  significant  part  in  group  interaction.  These 
include  distance  between  group  members,  physical  contact  (touch),  visual  orientation 
(eye  contact),  and  facial  expression  (especially  smihng)  (McGrath,  1984).  Each  of  these 
nonverbal  actions  plays  a  role  in  the  group  communication  process. 

Anonymity  can  facilitate  group  processes  by  moderating  those  who  dominate 
group  discussions  (decision  by  minority  rule),  have  a  high  position  in  the  group  (decision 
by  authority  method),  and  rely  on  nonverbal  cues  to  get  their  point  across.  Hiltz  and 
Turoff  (1978)  note  that  computer-based  media  yield  a  notion  of  impersonality  and  that 
this  effect  is  further  heightened  by  user  anonymity.  Some  researchers  have  hypothesized 
that  anonymity  enhances  group  member  participation  by  reducing  inhibitions  (DeSanctis 
and  Gallupe,  1987). 

Numerous  studies  have  compared  participation  rates  between  GSS  aided  groups 
and  face-to-face  groups.  Many  studies  (Lewis,  1982;  Zigurs,  Poole  and  DeSanctis,  1988; 
George,  Easton,  Nunamaker  and  Northcraft,  1990;  Mejias  and  Shepherd,  1996)  have 
found  that  the  GSS  reduces  individual  dominance  and  that  there  is  a  more  equal 
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distribution  of  comments.  However,  it  was  further  discovered  that  “there  is 
unquestionably  a  greater  amount  of  communication  flow  during  a  face-to-face  conference 
than  during  a  computerized  conference  that  lasts  the  same  amount  of  time  (Hiltz,  Johnson 
and  Turoff,  1986:  236).”  They  found  that  every  face-to-face  group  had  more  comments 
than  every  GSS  group.  This  can  be  attributed  to  the  fact  that  people  can  talk  much  faster 
then  they  can  type.  This  may  also  be  attributed  to  people  trying  to  avoid  long  periods  of 
silence. 

One  method  through  which  anonymity  can  be  manipulated  is  comment  labeling. 
Comment  labeling  removes  content  anonymity.  In  a  labeled  environment,  the 
participant’s  comments  are  labeled  with  a  unique  identifier.  In  an  unlabeled 
environment,  the  participant’s  comments  are  not  labeled  so  their  comments  are  not 
distinguishable  from  other  participant’s  comments.  When  participation  is  labeled,  either 
through  comment  labeling  or  in  face-to-face  meetings,  participants  receive  real-time 
feedback  on  the  performance  and  contributions  of  other  group  members.  Labeling 
comments  can  impact  the  group  process  both  positively  and  negatively.  On  the  negative 
side,  labeling  can  obstruct  some  users  who  require  anonymity  in  order  to  participate  in  a 
meeting.  On  the  positive  side  comment  labeling  provides  real-time  feedback  to  group 
members  and  may  reduce  user  tendencies  to  socially  loaf,  while  concurrently  providing 
information  on  participation  rates  to  the  entire  group. 

Of  the  ten  comment  labeling  studies  included  in  Table  3,  three  found  no 
difference  in  the  participation  rate  between  unlabeled  and  labeled  GSS  groups,  but  the 
other  seven  studies  all  found  a  higher  level  of  participation  in  unlabeled  (anonymous) 
groups. 
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Wilson  and  Jessup  (1995)  propose  that  anonymous  GSS  groups  should  allow 
more  ideas  to  be  generated  during  a  meeting,  because  group  members  with  low-status  can 
contribute  ideas  more  freely  and  openly.  Wilson  and  Jessup’s  study  found  that 
anonymous  GSS  groups  generated  more  total  coimnents  than  did  identified  GSS  groups 
(1995).  Jessup  et  al.  (1990)  found  a  higher  level  of  comments  contributed  in  anonymous 
GSS  groups.  They  determined  that  “by  disassociating  individuals  from  their  comments 
and  buffering  group  members  from  one  another,  anonymity  appears  to  have  reduced 
behavioral  constraints  on  group  members  and  led  them  to  contribute  more  freely,  and  less 
inhibitedly,  to  the  group  (Jessup  et  al.,  1990:  320).” 

Hypothesis  1:  Effect  of  anonymity  on  user  participation 

The  level  of  anonymity  provided  in  a  problem-solving  environment  (GSS  only) 
will  positively  impact  user  participation  levels;  however,  face-to-face  groups  will  achieve 
the  highest  participation  levels  (see  Figure  3). 

Hypothesis  la:  Face-to-face  groups  will  achieve  the  higher  levels  of  user 
participation  then  all  GSS  treatment  groups. 


Hypothesis  lb:  The  level  of  anonymity  in  GSS  environment  will  positively  impact 
user  participation  levels. 
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Figure  3.  Effect  of  anonymity  on  user  participation  rate. 


31 


2.6.2  Influence  of  Anonymity  on  Decision  Quality 

One  of  the  benefits  of  anonymity  is  that  it  may  reduce  the  pressure  to  conform  to 
the  groups  thought  process  and  minimize  evaluation  apprehension.  These  process  gains 
are  often  tempered  by  an  increase  in  free  riding  because  it  is  more  difficult  to  determine 
when  someone  is  free  riding  (Albanese  and  VanHeet,  1985).  Other  benefits  attributed  to 
anonymity  and  ultimately  resulting  in  improved  decision  quality  include  more  objective 
evaluation  and  the  creation  of  a  low-threat  environment  (Nunamaker  et  al.,  1993). 
Anonymity  may  encourage  participants  to  challenge  each  other  resulting  in  process  gains 
(catching  errors  before  the  group  decision).  The  low-threat  environment  created  in  an 
anonymous  meeting  can  also  increase  the  number  of  contributions  from  less  skilled 
members  (Nunamaker  et  al.,  1993). 

Of  the  studies  included  in  table  2,  five  (Steeb  and  Johnston,  1981;  Lewis,  1982; 
Gallupe  and  DeSanctis,  1988;  Zigurs  et  al.,  1988;  Jarvenpaa,  Rao  and  Huber,  1988)  found 
that  a  higher  quality  decision  was  reached  in  GSS  aided  groups.  Eight  studies  found  no 
effect  on  decision  quality. 

Gallupe  and  DeSanctis  (1988)  found  that  a  GSS  improved  decision  quality  for 
both  low  and  high  difficulty  decision  tasks.  They  found  that  the  GSS  acted  as  the  “group 
memory”  and  enabled  the  group  to  analyze  information  without  unnecessarily  repeating 
the  analysis.  This  in  turn  eillowed  the  group  to  consider  more  alternatives  eind  improved 
decision  quality  (Gallupe  and  DeSanctis,  1988). 

Hiltz  et  al.  (1986)  determined  decision  quality  by  comparing  the  group  score  to 
the  mean  of  the  individual  choices  before  the  discussion  and  with  the  best  individual 
decision  before  the  group  discussion.  They  found  no  significant  differences  in 
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percentage  of  improvement  in  the  sixteen  groups  evaluated.  Of  these  groups,  twelve 
produced  an  equal  or  better  decision  as  a  group  after  discussion  than  any  of  their 
individual  members  before  the  discussion.  Two  of  the  groups  that  produced  poorer- 
quality  decisions  were  face-to-face  groups  and  two  were  GSS  groups  (Hiltz  et  al.,  1986). 

Only  two  studies  compared  decision  quality  between  labeled  GSS  and  unlabeled 
GSS  groups.  One  (Valacich,  Dennis  and  Nunamaker,  1992)  found  no  difference  and  the 
other  (Jessup  et  al.,  1990)  found  higher  quality  decisions  were  reached  in  unlabeled 
(anonymous)  GSS  sessions. 

Using  the  chronic  campus  parking  problem  as  the  focal  point  of  their  problem¬ 
solving  group,  Jessup  et  al.  (1990)  found  that  groups  working  anonymously  generated 
more  solutions  to  the  problem  then  did  identified  groups. 

The  process  gains  (reduced  evaluation  apprehension,  more  errors  caught,  low- 
threat  environment)  outweigh  the  process  losses  (free  riding)  in  an  anonymous  group 
problem-solving  enviromnent.  Subsequently,  based  on  the  literature  findings  and  the 
process  gains  associated  with  increased  anonymity  the  quality  of  group  decisions  should 
improve  when  participants  are  provided  with  a  greater  level  of  anonymity. 

Hypothesis  2:  Effect  of  anonymity  on  group  decision  quality 

The  level  of  anonymity  provided  in  a  problem-solving  environment  (either  GSS 
or  face-to-face)  will  positively  impact  group  decision  quality  (see  Figure  4). 
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Figure  4.  Effect  of  anonymity  on  group  decision  quality. 


2.6.3  Influence  of  Anonymity  on  Consensus  Making 

“In  the  case  of  consensus  decision  tasks,  decision  theorists  propose  that  groups 
can  solve  complex  problems  more  effectively  if  their  discussion  includes  high  member 
participation  and  a  decision-making  structure  (DeSanctis,  D’Onofrio,  Sambamurthy  and 
Poole,  1989:  132).”  However,  when  higher  levels  of  anonymity  are  provided  to 
participants  the  discussion  tends  to  be  “more  open,  honest,  and  free-wheeling” 
(Nunamaker  et  al.,  1993).  Often  increasing  levels  of  anonymity  can  result  in  flaming. 
Flaming  is  where  group  members  begin  to  demonstrate  uninhibited  interaction  and  begin 
to  send  critical  comments  to  other  members  of  the  group.  Flaming  occurs  when 
anonymous  group  members  begin  to  lose  their  individuality  which  results  in  a  reduction 
of  normal  inhibitions,  enabling  group  members  to  engage  in  behavior  they  would  not 
normally  display  in  a  labeled  situation  (Jessup  and  Connolly,  1990). 

Among  the  studies  included  in  table  2,  six  found  that  consensus  was  either  less 
likely  in  a  GSS  environment  or  that  a  lower  level  of  consensus  was  likely  in  a  GSS 
environment.  In  two  studies  (Olaniran,  1994;  Dennis,  1996)  that  evaluated  time  to  reach 
consensus,  it  was  found  that  it  takes  longer  to  reach  consensus  in  a  GSS  environment.  In 
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only  one  study  (Mejias  and  Shepherd,  1996)  was  their  no  difference  between  face-to-face 
and  GSS  sessions  when  striving  to  achieve  consensus. 

Turoff  and  Hiltz  (1982)  found  that  there  is  more  opinion  giving  in  computerized 
conferencing  and  that  this  negatively  impacts  the  ability  to  reach  consensus.  In  their 
study,  Turoff  and  Hiltz  (1982)  found  that  half  of  face-to-face  groups  successfully  reached 
100  percent  consensus  while  none  of  the  computerized  conference  groups  did. 

Watson,  DeSanctis  and  Poole  (1988)  related  post-meeting  consensus  to  pre¬ 
meeting  consensus  and  determined  that  in  GSS  conditions  group  members  apparently 
became  aware  of  their  pre-meeting  agreement  and  differences  and  subsequently  reached  a 
higher  level  of  post-meeting  consensus  than  did  face-to-face  groups. 

Despite  the  findings  presented  by  Watson  et  al.  (1988),  many  attributes  of 
anonymity  leads  one  to  believe  that  anonymity  will  have  an  adverse  effect  on  consensus. 
The  fact  that  increased  opinion-giving  negatively  relates  to  consensus  and  the  fact  that 
anonymity  reduces  conformance  pressures  (occasionally  resulting  in  flaming)  indicates 
that  increasing  the  level  of  anonymity  in  a  problem-solving  group  should  result  in  lower 
levels  of  consensus. 

Hypothesis  3:  Effect  of  anonymity  on  group  consensus 

The  level  of  anonymity  provided  in  a  problem-solving  environment  (either  GSS 
or  face-to-face)  will  negatively  impact  group  consensus  (see  Figure  5). 
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Figure  5.  Effect  of  anonymity  on  group  consensus. 


2.6.4  Influence  of  Anonymity  on  User  Attitudes 

Despite  the  risk  of  flaming,  there  are  some  positive  effects  from  depersonalization 
provided  by  increased  anonymity.  For  instance,  anonymity  increases  objective 
evaluation.  In  this  case,  contributions  are  judged  based  on  their  merits  rather  then  on  the 
source  of  the  contribution.  Criticism  is  perceived  as  being  directed  at  the  idea,  not  the 
contributor  (Nunamaker  et  al.,  1993). 

The  findings  on  satisfaction  with  the  process  when  comparing  GSS  sessions  to 
face-to-face  sessions  is  contradictory.  Two  studies  found  GSS  groups  to  be  more 
satisfied  (Steeb  and  Johnston,  1981;  Nunamaker  et  al.,  1988),  two  found  higher 
satisfaction  levels  in  face-to-face  groups  (Gallupe  and  DeSanctis,  1988;  Watson  et  al., 
1988),  and  five  found  no  differences  (Lewis,  1982;  Jarvenpaa  et  al.,  1988;  Easton,  1988; 
George  et  al.,  1990;  Dennis,  1996). 

Watson  et  al.  (1988)  found  that  GSS  groups  appeared  to  struggle  with  how  to 
effectively  use  the  GSS  technology.  They  also  found  that  GSS  groups  often  entered 
comments  such  as  “What  do  you  want  to  do  next?  (Watson  et  al.,  1988:  474).”  This  lack 
of  GSS  knowledge  and  meeting  direction  may  explain  the  improved  user  attitudes  in  the 
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face-to-face  sessions.  Including  training  and  a  structured  agenda  might  mitigate  this 
perception. 

Nunamaker  et  al.  (1988)  found  that  despite  some  difficulty,  their  participants 
reported  high  levels  of  satisfaction  with  the  computer  technology  for  generating  ideas 
during  organization  planning  sessions.  They  further  found  that  “while  there  are  some 
factors  inherent  in  the  technology  that  tend  to  inhibit  idea  generation,  the  benefits  of  the 
technology  appear  to  far  outweigh  the  problems  (Nunamaker  et  al.,  1988:  18).” 

When  comparing  GSS  labeled  to  GSS  unlabeled  research  on  satisfaction  with  the 
process  the  results  are  once  again  contradictory.  Two  studies  found  labeled  GSS  groups 
to  be  more  satisfied  with  the  process  (Jessup  and  Tansik,  1991;  Valacich  et  al.,  1992), 
one  found  unlabeled  groups  to  be  more  satisfied  (Kahai,  Avolioi  and  Sosk,  1995),  and 
two  found  no  effect  (Connolly  et  al.,  1990;  Jessup  et  al.,  1990). 

Valacich  and  Nunamaker  (1992)  found  that  labeled  groups  were  both  more 
satisfied  with  the  process  and  felt  more  effective  than  anon)mious  groups.  Jessup  and 
Tansik  (1991)  found  that  group  members  working  under  conditions  of  anonymity  thought 
the  system  was  more  helpful  and  effective.  Nunamaker,  Dennis,  Valacich  and  Vogel 
(1991)  proposed  that  anonymity  reduces  satisfaction  because  it  makes  it  difficult  for 
participants  to  get  credit  for  their  comments. 

Jessup  and  Connolly  (1990)  found  that  users  felt  more  satisfied  with  the 
process  in  an  unlabeled  GSS  environment  because  they  felt  criticism  was  aimed  at 
their  comments  rather  than  them  personally.  Kahai  et  al.  (1995)  found  that 
anonjunity  increases  user  satisfaction  and  attributed  this  to  the  fact  that  in  a  meeting 
where  little  differences  of  opinion  exist,  there  will  be  less  evaluation  apprehension 
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and  less  critical  remarks.  Subsequently  the  users  attitudes  will  increase  in  the 
anonymous  condition. 

Finally,  despite  finding  no  effect  of  anonymity  on  user  satisfaction  with  the 
process,  Wilson  and  Jessup  (1995)  hypothesized  that  anonymous  group  members  should 
be  more  satisfied  than  identified  group  members,  because  anonymous  group  members 
will  be  able  to  contribute  ideas  more  freely  and  openly  than  would  identified  group 
members. 

Despite  the  contradictory  findings  listed  in  Tables  2  and  3  it  is  hypothesized  that 
increased  anonymity  should  improve  user  attitudes  because  of  the  depersonalization  of 
the  comments  and  ultimately  the  critiques  of  those  comments. 

Hypothesis  4:  Effect  of  anonymity  on  user  attitudes 

The  level  of  anonymity  provided  in  a  problem-solving  environment  (either  GSS 
or  face-to-face)  will  positively  impact  user  attitudes  (see  Figure  6). 
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Figure  6.  Effect  of  anonymity  on  user  attituc 

les. 

2.6.5  Influence  of  Anonymity  on  User  Satisfaction  with  the  Group  Outcome 

A  study  by  Rao  and  Monk  (1999)  stated  that  if  participants  made  a  decision 
anonymously,  the  need  for  external  justification  would  not  exist  and  the  participants  level 
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of  commitment  to  the  group  decision  would  be  lower  than  the  eommitment  of  identified 
participants  because  the  identified  participants  would  require  external  justification.  If  the 
participants  are  anonymous,  then  they  only  “need  to  maintain  an  internal  sense  of 
competence  (Rao  and  Monk,  1999:  502).”  This  difference  in  desire  to  appear  externally 
competent  results  in  a  higher  level  of  commitment  when  participants  are  identified. 

Table  2  depicts  the  differences  in  satisfaction  with  outcome  findings  in  previous 
face-to-face  and  GSS  studies.  Of  the  six  studies  included,  two  found  no  effect  on 
satisfaction  with  the  group  outcome  between  GSS  groups  and  face-to-face  groups.  Two 
found  a  higher  level  of  satisfaction  in  GSS  groups  and  two  found  the  GSS  reduced  the 
satisfaction  level  of  participants. 

Steeb  and  Johnston  used  a  group  decision  aiding  system,  which  aided  users  with 
decision  tree  structuring,  identification  of  critical  issues,  conflict  resolution,  and  decision 
recommendation.  Unaided  users  (face-to-face)  were  given  paper,  pencils  and  a 
blackboard.  Their  study  found  that  seventy-seven  percent  of  the  aided  users  fully 
supported  the  team’s  chosen  course  of  action  and  only  fifty-three  percent  of  the  unaided 
users  fully  supported  their  group’s  decision  (Steeb  and  Johnston,  1981).  Easton  (1988) 
found  that  GSS  groups  using  stractured  approaches  were  more  satisfied  with  the  process 
and  ultimately  were  more  satisfied  with  the  outcome. 

Gallupe  and  DeSanctis  (1988)  found  groups  that  used  the  GSS  were  slightly  less 
confident  in  the  decision  they  had  made  and  had  a  slightly  lower  level  of  agreement  with 
the  final  solution  compared  to  face-to-face  groups.  Positive  sentiments  following  use  of  a 
GSS  may  be  particularly  important  in  organizational  settings  where  group  meetings  are 
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used  to  gain  acceptance  of  ideas,  rather  than  to  gain  better  ideas  per  se  (Gallupe  and 
DeSanctis,  1988). 

Only  one  study,  which  compared  labeled  GSS  sessions  with  unlabeled  GSS 
sessions,  reported  findings  in  the  area  of  satisfaction  with  the  group  outcome.  Valacich  et 
al.  (1992)  found  that  labeled  groups  felt  more  effective  than  anonymous  groups.  They 
further  found  that  the  identified  condition  most  closely  resembled  a  traditional  face-to- 
face  meeting,  thereby  enhancing  the  member’s  perceptions  of  their  effectiveness  in  this 
condition  (Valacich  et  al.,  1992). 

Hypothesis  5:  Effect  of  anonymity  on  user  satisfaction  with  the  group  outcome 

The  level  of  anonymity  provided  in  a  problem-solving  environment  (either  GSS 
or  face-to-face)  will  negatively  impact  user  satisfaction  with  the  group  outcome  (see 
Figure  7). 


Figure  7.  Effect  of  anonymity  on  user  satisfaction  with  group  outcome. 


2.7  Summary 

The  GSS  has  been  offered  up  as  the  solution  for  effectively  improving  group 
processes  to  improve  user  participation  and  decision  quality.  Many  proponents  of  GSS 
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enthusiastically  advertise  the  ability  of  the  technology  to  improve  the  generation  and 
sharing  of  information  (Nunamaker,  1997).  However,  the  studies  described  in  this 
chapter  indicate  that  GSS  may  not  be  the  best  option.  Current  GSS  research  presents 
contradictory  results  and  indicates  most  clearly,  the  need  for  further  research. 

The  next  chapter  describes  the  methodology  used  to  determine  the  influence  of 
process  anonymity  and  content  anonymity  on  participation  rates,  decision  quality, 
consensus  levels,  user  satisfaction  with  the  process,  and  user  satisfaction  with  the  group 
outcome. 
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in.  Methodology 


3.1  Overview 

The  purpose  of  this  study  was  to  determine  what  effect  anonymity  might  have  on 
group  members’  perceptions  of  the  problem-solving  environment,  group  consensus  and 
ultimately  the  group  decision  quality.  The  previous  chapter  summarized  the  relevant 
research  literature  and  expanded  on  the  theoretical  framework  developed  in  the  area  of 
Group  Support  System  anonymity.  This  chapter  describes  how  data  were  collected; 
quantified,  and  analyzed  to  test  the  hypotheses  described  in  Chapter  n. 

3.2  Participants 

The  320  subjects  who  participated  in  this  study  were  drawn  from  the  graduate 
student  body  of  the  Air  Force  Institute  of  Technology  (n=48),  students  enrolled  in  the 
Basic  Communications  and  Information  Officer  Course  (BCOT)  at  Keesler  AFB, 
Mississippi  (n=216)  and  Reserve  Officer  Training  Corps  (ROTC)  students  enrolled  at  the 
University  of  Pittsburgh  in  Pennsylvania  (n=32),  Ohio  University  in  Athens,  Ohio 
(n=16),  and  Wright  State  University  in  Fairborn,  Ohio  (n=8).  The  age  of  the  participants 
ranged  from  18  to  53  years  old.  Table  4  below  presents  the  study’s  demographic 
breakdown.  Their  military  ranks  ranged  from  ROTC  Cadet  4*  class  to  commissioned 
officers  through  the  rank  of  Lieutenant  Colonel.  The  treatments  were  applied  randomly 
across  all  participants  to  ensure  that  no  individual  group  of  participants  from  one  location 
could  bias  the  results. 
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Table  4. 


Demographic  Breakdown 


Age  (in  years) 

Participants 

Percentage 

<20 

29 

9% 

20-29 

205 

64% 

30-39 

74 

23  % 

40-49 

10 

3% 

>50 

2 

1% 

Sex  (Gender) 

Participants 

Percentage 

Male 

254 

79% 

Female 

66 

21  % 

Marital  Status 

Participants 

Percentage 

Married 

144 

45% 

Single 

176 

55% 

3.3  Experimental  Design 

This  study  used  a  fully  randomized  experiment  to  investigate  the  effect  of 
anonymity  on  the  perceptions,  attitudes,  and  subsequent  performance  of  all  members  of  a 
four-person  problem-solving  team.  A  1  x  4  between  subjects  experimental  design 
manipulated  the  level  of  anonymity  experienced  by  each  of  the  participants  through  the 
use  of  GSS  comment  labeling  and  placards  (see  Table  5). 
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Table  5. 


Level  of  Anonymity  in  comparison  to  Experimental  Treatment 


hlSBISBm 

Treatment 

Very  High 

Unlabeled  GSS  Session 

High 

Labeled  GSS  Session  without  Placards 

Low 

Labeled  GSS  Session  with  Placards 

Very  Low 

Face-to-face  Session 

As  can  be  seen  in  Table  5  the  highest  level  of  anonymity  is  provided  in  unlabeled 
GSS  sessions  because  each  comment  input  into  the  system  cannot  be  associated  with  any 
group  member.  The  labeled  GSS  sessions  without  placards  follow  closely  behind  since 
comments  are  labeled  by  colors,  but  the  colors  cannot  be  associated  with  any  single 
group  member.  This  only  allows  group  members  to  mentally  group  comments  labeled 
with  the  same  color.  Both  GSS  sessions  labeled  with  placards  and  face-to-face  sessions 
provide  low  levels  of  anonymity,  however,  associating  a  comment  with  the  member  that 
provided  it  orally  is  instantaneous  thus  reducing  the  anonymity  even  further.  In  the  GSS 
sessions  labeled  with  placards  the  members  apply  a  three-step  process  to  correlate 
comments  with  the  originator.  The  member  has  to  first  look  at  the  comment,  then 
identify  the  color  at  the  end  of  the  comment  and  match  that  with  the  terminal  it  came 
from  and  ultimately  the  member  sitting  behind  that  terminal. 

When  compared  to  anonymity  in  the  face-to-face  condition,  the  factors  of 
explicitness  and  revocability  must  be  applied  to  the  labeled/placard  condition. 
Explicitness  is  defined  as  “the  extent  to  which  an  action  can  be  said  to  have  taken  place” 
(Salancik,  1995:  285).  In  a  GSS  labeled  with  placard  session  subjects  comments  are 
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clearly  observable  and  associated  with  the  sender.  In  a  face-to-face  meeting  a  subject  can 
deny  having  made  a  comment  if  the  session  was  not  recorded.  Salancik  defines 
revocability  as  the  “reversibility  of  the  action”  (1995:  285).  Contrary  to  a  face-to-face 
session,  a  comment  input  in  a  GSS  labeled  with  placard  session  remains  on  the  screen  for 
all  to  see.  It  cannot  be  reversed. 

One  other  significant  difference  between  these  treatments  is  the  different  human 
senses  utilized.  In  face-to-face  sessions,  members  use  sight  and  hearing  simultaneously 
to  assign  comments  to  the  sender.  In  all  three  GSS  treatments,  the  member  is  confined  to 
one  sense  (sight).  Thus,  the  ability  to  receive  information  through  non-verbal  clues  (such 
as  facial  expressions)  and  tone  of  voice  is  minimized  in  a  GSS  environment. 

3.4  Experiment  Manipulations 

Two  experiment  manipulations  were  used  in  this  study:  process  anonymity  and 
content  anonymity  (comment  labeling).  To  manipulate  content  and  process  anonymity, 
subjects  were  randomly  assigned  to  face-to-face  groups,  labeled  with  placard  GSS 
groups,  labeled  without  placard  GSS  groups,  or  unlabeled  GSS  groups.  At  the  conclusion 
of  each  session,  a  survey  (questions  34  -  39)  measuring  the  effectiveness  of  the 
manipulation  checks  was  given.  This  survey  is  attached  in  Appendix  A.  ANOVA  was 
used  to  assess  the  effectiveness  of  a  manipulation,  by  comparing  the  means  of  survey 
data  collected  from  groups  which  experienced  the  manipulations  to  those  which  did  not. 
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3.4.1  Anonymity  Manipulation 

Four  levels  of  anonymity  were  created  through  the  structural  design  of  the  group 
session.  A  consolidated  series  of  comment-member  association  steps  for  each  level  of 
anonymity  is  listed  in  Table  6. 


Table  6. 


Association  Steps  for  each  Level  of  Anonymity 


GSS -Unlabeled 

GSS -Labeled- 

GSS -Labeled- 

Face-to-face 

No  Placard 

Placard 

Anonymity  Provided 

Content  Anonymity 

Process  Anonymity 

None 

None 

Process  Anonymity 

Associated  Steps 

1.  Read  comment. 

1.  Read  comment 

1.  Read  comment 

1. 

Hear  comment. 

and  color. 

and  color. 

2.  Color  and  placard 

2. 

See  member 

not  available  so 

2.  Placard 

2.  Identify  placard 

speak. 

comment  cannot 

unavailable  so 

with  comment 

be  associated  with 

comment  is 

color. 

3. 

Associate 

member  or  color. 

associated  with 

comment  with 

previous 
comments  made 

3.  Associate  placard 
with  member. 

member. 

by  same  color. 

4.  Associate 

comment  with 

- 

member. 

GSS  anonymity  was  manipulated  by  using  the  labeling  feature  provided  in  Group 
Systems  software.  The  highest  level  of  anonymity  provided  was  accomplished  by 
leaving  all  subjects  as  unlabeled  participants.  In  this  case,  only  the  time  stamp  followed 
comments.  No  information  was  available  for  participants  to  group  comments. 
Subsequently,  other  session  participants  could  not  identify  who  made  each  comment  and 
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could  not  group  comments  by  color.  Participants  in  this  condition  experienced  both 
process  and  content  anonymity. 

The  next  level  of  anonymity  was  accomplished  by  only  labeling  each  participant’s 
comments.  In  this  case,  the  time  stamp  was  accompanied  by  the  subject’s  color  (red, 
green,  yellow,  and  blue).  Each  session  participant  could  identify  which  comments 
belonged  to  each  “color”.  This  would  allow  them  to  associate  one  comment  with  another 
from  the  same  source,  but  did  not  allow  them  to  “put  a  face  with  each  comment.”  This 
condition  offered  no  content  anonymity. 

The  lowest  level  of  GSS  anonymity  labeled  subjects  and  placed  a  placard 
identifying  their  color  (red,  green,  yellow,  and  blue)  above  their  computer  terminal.  In 
this  treatment,  the  placard  allowed  each  participant  to  put  a  face  with  the  comments  by 
associating  the  color  who  made  the  comment  with  the  placard  above  the  monitor  and 
ultimately  with  the  participant  using  that  monitor.  This  condition  contained  no  content 
or  process  anonymity. 

Finally,  a  quarter  of  the  groups  solved  the  problem  face-to-face  around  a  table. 
The  face-to-face  treatment  provided  virtually  no  anonymity  since  participants  sat  around 
a  table  facing  each  other  and  could  see  who  was  contributing. 

One  of  the  challenges  encountered  during  the  development  of  this  study  was  to 
ensure  both  GSS  and  face-to-face  groups  were  treated  as  similarly  as  possible  excluding 
the  manipulations  themselves.  Great  effort  was  put  forth  to  ensure  that  the  scripts  were 
as  close  to  identical  as  possible.  Where  discussion  revolved  around  the  computer 
technology  in  the  GSS  sessions,  similar  conversation  was  built  into  the  face-to-face 
script.  The  overhead  projection  of  the  team  votes  and  the  display  screen  shown  for  GSS 
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groups  was  replicated  and  projected  on  a  whiteboard  for  face-to-face  groups.  Every 
effort  was  made  to  ensure  the  four  groups  had  similar  experiences  during  the  sessions  so 
the  focus  could  be  on  the  manipulations.  Further  details  for  each  configuration  will  be 
explained  in  the  following  sections. 

Table  7. 

Reliability  Analysis — Anonymity  Manipulation  Checks 

M  SD  a 

Manipulation  Check  1;  Anonymity  5.10  1.59  0.85 

I  could  recognize  the  originator  of  most  comments.  5.00  1.74 

Other  group  members  could  connect  me  to  the  comments  I  made.  5.03  1 .54 

Other  group  members  knew  when  I  made  a  contribution  to  the  group.  5.26  1 .48 

A  one-way  analysis  of  variance  compared  the  mean  anonymity  manipulation 
checks  of  decision  making  groups  using  unlabeled  GSS  sessions,  labeled  without  placard 
GSS  sessions,  labeled  with  placard  GSS  sessions,  and  face-to-face  sessions.  The  alpha 
level  was  .05.  This  test  was  found  to  be  statistically  significant,  F(3,  268)  =  39.53,  p  < 

.01.  The  strength  of  the  relationship,  as  indexed  by  eta^,  was  .31.  A  multiple  comparison 
was  accomplished  using  the  Bonferroni  procedure  to  control  for  experiment-wise  error. 

The  results  of  this  test  indicated  that  the  mean  for  unlabeled  GSS  sessions  (M  =  3.77,  SD 
=  1.30)  was  statistically  (p  <  .01)  less  than  the  mean  for  labeled  without  placard  GSS 
sessions  (M  =  5.12,  SD  =  1.30).  In  addition,  the  mean  for  labeled  without  placard  GSS 
sessions  was  statistically  (p  <  .05)  less  than  the  mean  for  both  labeled  with  placard  GSS 
sessions  (M  =  5.70,  SD  =  1.21)  and  face-to-face  sessions  (M  =  5.71,  SD  —  .77).  There 
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was  no  statistically  reliable  difference  between  the  mean  for  labeled  with  placard  GSS 
sessions  and  face-to-face  sessions.  The  results  indicate  the  manipulation  to  create 
process  anonymity  was  very  effective  for  the  unlabeled  groups  and  less  effective  for  the 
labeled  without  placard  group. 

3.4.2  Comment  Labeling 

The  labeling  manipulation  check  was  accomplished  in  the  same  manner  as  the 
anon5nnity  manipulation  check  as  described  in  paragraph  3.4.1.  Most  participants  had 
never  used  a  GSS  before  the  experiment  and  likely  did  not  know  they  could  be  labeled  or 
unlabeled.  In  four  unlabeled  GSS  sessions  participants  actually  decided  to  label 
themselves,  thus  circumventing  the  labeling  process  provided  in  the  GSS  environment. 
These  participants  indicated  they  wanted  to  know  who  was  making  the  conunents  during 
the  session.  After  the  fourth  self-labeling  session,  the  script  was  modified  asking 
participants  not  to  label  themselves  and  the  four  self-labeling  sessions  were  eliminated 
from  the  experiment  data  set. 

A  one-way  analysis  of  variance  compared  the  mean  comment  labeling 
manipulation  checks  of  decision  making  groups  using  unlabeled  GSS  sessions,  labeled 
without  placard  GSS  sessions,  labeled  with  placard  GSS  sessions,  and  face-to-face 
sessions.  The  alpha  level  was  .05.  This  test  was  found  to  be  statistically  significant,  F(3, 
268)  =  41.51,  p  <  .01.  The  strength  of  the  relationship,  as  indexed  by  eta^,  was  .32.  A 
multiple  comparison  was  accomplished  using  the  Bonferroni  procedure  to  control  for 
experiment-wise  error.  The  results  of  this  test  indicated  that  the  mean  for  unlabeled  GSS 


49 


Table  8. 


Reliability  Analysis  —  Comment  Labeling  Manipulation  Checks 


M 

SD  a 

Manipulation  Check  2:  Comment  Labeling 

4.87 

1.63  0.88 

I  could  tell  if  someone  was  sharing  more  information  than  other  members 

4.85 

1.62 

of  the  group  were. 

I  could  tell  if  someone  participated  less  than  other  members  of  the  group 

4.74 

1.70 

did. 

Other  group  members  could  judge  the  extent  that  I  participated  in  the 

5.02 

1.57 

group. 

sessions  (M  =  3.41, 5D  =  1.38)  was  statistically  (p<  .01)  less  than  the  mean  for  labeled 
without  placard  GSS  sessions  (M  =  5.17,  5D  =  1.1 1),  labeled  with  placard  GSS  sessions 
(M  =  5.21,  SD  =  1.29),  and  face-to-face  sessions  (M  =  5.60,  SD  =  1.10).  There  was  no 
statistically  reliable  difference  between  the  mean  for  labeled  without  placard  GSS 
sessions,  labeled  with  placard  GSS  sessions,  and  face-to-face  sessions.  The  results 
indicate  that  the  manipulation  to  create  content  anonymity  in  the  unlabeled  group  was 
effective. 

3.5  Equipment  and  Room  Configuration 

All  experiment  sessions  were  conducted  in  rooms  with  no  windows.  Rooms  were 
reserved  for  the  study  to  minimize  any  chance  of  interruptions.  Each  participant  had  a 
chair  and  a  desk  surface.  GSS  teams  used  PC-type  computers  loaded  with  Group 
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Systems  software,  developed  by  Ventana  Corporation  (now  known  as 
GroupSystems.com).  Face-to-face  teams  were  provided  with  pencil  and  paper,  as  well  as 
a  white  board  with  markers.  Face-to-face  and  GSS  teams  were  trained  and  shown  the 
results  of  group  votes  using  an  overhead  projector.  Excel  charts  were  developed  to 
mirror  the  look  of  results  from  GSS  sessions. 

A  single  GSS  room  configuration  (Appendix  B)  was  used  in  all  fifty-two  GSS 
sessions.  Special  effort  was  made  to  ensure  the  distance  between  user  terminals  was  the 
same.  Approximately  four  feet  separated  subjects  in  order  to  minimize  the  observation  of 
other  subjects’  monitors.  For  sessions  requiring  placards,  they  were  placed  on  top  of 
each  participant’s  computer  monitor  making  them  highly  visible.  In  addition,  in  sessions 
where  a  placard  was  used  the  participants’  attention  was  directed  to  the  location  of  the 
placards.  All  face-to-face  sessions  were  conducted  in  rooms  configured  like  the  primary 
GSS  room  configuration  minus  the  PC-type  computers  for  the  subjects.  The  face-to-face 
configuration  is  depicted  in  Appendix  C.  In  the  GSS  and  face-to-face  room 
configurations,  all  participants  had  easy  viewing  access  of  the  viewing  screen.  Since  the 
screen  was  only  used  during  the  training  session  and  when  displaying  the  group  vote  this 
minimized  the  impact  of  any  possible  differences  between  the  two  configurations. 

3.6  GSS  Tasks  and  Procedures 

The  tasks  and  procedures  for  all  GSS  groups  were  identical  with  only  two  notable 
exceptions.  Before  each  GSS  session,  the  experiment  equipment  was  configured  for  one 
of  the  three  types  of  GSS  sessions  (unlabeled,  labeled  without  placard,  and  labeled  with 
placard)  according  to  the  two  GSS  scripts  (GSS  labeled  and  GSS  unlabeled).  In  the 
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unlabeled  session,  the  GSS  software  was  configured  to  ensure  participant’s  conunents 
were  unlabeled.  In  the  labeled  without  placard  and  labeled  with  placard  sessions  the  GSS 
software  was  configured  to  ensure  peirticipant’s  conunents  were  labeled.  The  script  was 
modified  for  the  labeled  scripts  to  point  out  that  the  comments  had  a  color  after  each 
comment.  The  script  was  modified  slightly  more  for  the  labeled  with  placard  sessions  to 
allow  the  participants  to  recognize  the  location  of  the  placards  and  how  they 
corresponded  to  the  color  after  each  comment.  These  scripts  are  included  in  Appendix  D 
and  E.  The  experiment  procedural  flow  is  depicted  in  Figure  8. 

Four  experimental  subjects  interacted  with  two  experiment  administrators  using  a 
GSS  running  Group  Systems  software.  One  experiment  administrator  was  the  group 
facilitator  and  the  other  was  the  assistant.  Both  administrators  worked  from  scripts.  Four 
subjects  were  scheduled  for  each  session  and  sessions  were  not  held  unless  all  four 
subjects  were  available. 

Subjects  were  welcomed  in  the  preparation  room  (separate  from  the  room  with  the 
GSS  equipment).  Subjects  were  instructed  to  read  and  sign  a  consent  form  (see 
Appendix  F)  stating  their  rights  as  an  experiment  participant.  Consent  forms  were 
collected  by  the  experiment  assistant  and  placed  in  a  folder  separate  from  other 
experiment  information.  Next,  subjects  completed  an  introductory  personality 
questionnaire  (see  Appendix  G)  and  individually  completed  the  first  problem-solving  task 
(moon  scenario)  (see  Appendix  H).  Before  departing  the  preparation  room  for  the  GSS 
room,  the  subjects  received  a  brief  introduction  to  problem-solving  and  reaching 
consensus  through  voting. 
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Once  within  the  GSS  room,  subjects  were  allowed  to  pick  their  own  GSS  station 
(red,  yellow,  green  or  blue).  At  this  point,  the  experiment  assistant  documented  which 
participant  sat  at  each  station.  This  was  later  transferred  to  each  participant’s 
questionnaires  and  individual  problem-solving  exercises.  The  experiment  facilitator  then 
introduced  subjects  to  the  Group  System  software  package,  focusing  the  discussion  on 
two  tools  (Categorizer  and  Vote)  that  would  be  used  in  the  experiment.  The  subjects 
training  consisted  of  discussing  ranking  methods  for  a  list  of  names  and  departments 
using  the  Categorizer  tool  and  then  using  the  Vote  tool  to  rank  order  the  list.  The 
facilitator  used  this  training  time  to  familiarize  subjects  on  how  to  join  an  activity, 
approve  their  author  tag  (only  applicable  for  labeled  sessions),  input  comments, 
recognize  other  subjects’  comments  (whether  labeled  or  not),  and  vote.  Finally,  the 
facilitator  displayed  the  results  for  their  training  session  vote,  described  how  the  software 
determined  the  final  group  ranking,  and  discussed  the  results. 

Following  the  training,  experiment  administrators  instructed  subjects  that  they 
would  answer  any  questions  dealing  with  the  Group  Systems  software,  but  would  not 
answer  any  questions  dealing  with  the  moon  scenario.  Subjects  were  also  instracted  to 
limit  their  discussion  to  the  GSS  (no  verbal  comments)  so  all  comments  would  be 
collected.  Once  the  GSS  training  was  complete,  group  members  were  instructed  that  they 
would  now  have  fifteen  minutes  to  complete  the  moon  scenario  as  a  group.  After  a 
question  and  answer  period,  subjects  were  invited  to  join  the  moon  Categorizer  activity 
and  the  fifteen-minute  discussion  period  began.  Participants  were  not  given  access  to 
their  original  individual  solutions  although  a  few  participants  requested  them.  The 
experiment  facilitator  notified  subjects  when  there  were  two  and  five  minutes  left  to 
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discuss  the  scenario.  Once  the  subjects  exhausted  their  discussion  time,  the  experiment 
facilitator  instructed  the  subjects  to  submit  their  final  comment  (if  they  were  still  typing); 
close  their  group  discussion  window  and  the  facilitator  stopped  the  moon  scenario 
Categorizer  activity. 

Subjects  were  then  invited  to  join  a  voting  activity  for  the  moon  scenario.  All 
items  provided  in  the  voting  activity  were  alphabetized  to  replicate  the  list  in  the 
Categorizer  activity  and  in  the  individual  problem-solving  task.  Once  all  four  subjects 
had  cast  their  ballots,  the  experiment  facilitator  presented  the  results  to  the  group  and 
asked  the  group  if  all  members  were  willing  to  endorse  this  list  or  if  they  would  like  an 
additional  five  minutes  to  discuss  the  scenario.  If  all  subjects  endorsed  the  list,  subjects 
were  given  a  five-minute  break  and  were  instmcted  not  to  discuss  the  scenario  or  which 
participant  color  they  were  (only  applicable  to  labeled  sessions).  If  any  of  the  subjects 
preferred  to  continue  discussion,  then  all  subjects  were  re-invited  to  join  the  moon 
Categorizer  activity  where  subjects  could  review  previous  comments  or  provide  new 
comments.  The  experiment  facilitator  notified  subjects  when  there  were  two  minutes  left 
for  discussion.  Once  the  second  five-minute  discussion  period  was  complete,  subjects 
were  again  invited  to  join  a  voting  activity  for  the  moon  scenario.  Instead  of  receiving 
the  original  alphabetized  list  from  which  to  vote,  subjects  received  the  list  in  the  order  of 
their  previous  vote.  After  all  subjects  modified  their  vote  and  cast  their  ballots,  the 
facilitator  presented  the  final  results  of  their  vote,  subjects  were  given  a  five-minute  break 
and  subjects  were  instmcted  not  to  discuss  the  scenario  or  which  participant  color  they 
were  (only  applicable  to  labeled  sessions). 


55 


At  the  conclusion  of  their  break,  the  subjects  and  the  experiment  assistant  met 
back  in  the  preparation  room.  Subjects  first  completed  a  short  5-question  commitment  to 
ranking  questionnaire  (see  Appendix  I)  for  the  group  moon  results.  The  purpose  of  this 
questionnaire  was  to  quantify  the  level  of  confidence  each  participant  had  in  the  group’s 
final  decision  in  comparison  to  their  original  individual  ranking  and  also  to  gauge 
whether  individuals  truly  were  willing  to  endorse  the  group  decision. 

After  completing  the  questionnaire,  the  experiment  assistant  discussed 
participation  rate  goals  with  the  subjects  focusing  on  the  point  that  the  more  comments 
input  during  a  meeting,  the  greater  the  chances  to  reach  a  high  quality  decision.  At  this 
point,  half  of  the  groups  received  feedback,  which  showed  their  participation  rates  in  the 
moon  scenario.  Once  feedback  was  complete,  individuals  completed  the  second 
problem-solving  task  (desert  scenario)  (see  Appendix  J).  Before  departing  the 
preparation  room  for  the  GSS  room,  subjects  were  reminded  of  the  goal  to  participate 
equally  while  maximizing  comments. 

Once  back  in  the  GSS  room,  subjects  were  invited  to  join  the  desert  Categorizer 
activity  and  the  fifteen-minute  discussion  period  began.  At  this  point,  the  steps  for  the 
desert  GSS  activity  are  identical  to  the  steps  performed  above  for  the  moon  GSS  activity. 
After  all  subjects  modified  their  vote  and  cast  their  ballots,  the  facilitator  presented  the 
final  results  of  their  vote  and  the  experiment  assistant  accompanied  subjects  back  to  the 
preparation  room.  The  feedback  manipulation  and  subsequent  performance  and 
participation  data  collected  from  the  desert  activity  were  analyzed  by  one  of  the  other 
thesis  efforts  associated  with  this  experiment. 
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While  in  the  preparation  room,  subjects  completed  a  short  5-question  commitment 
to  ranking  questionnaire  for  the  group  desert  results.  Next,  subjects  completed  a  post¬ 
session  questionnaire,  which  assessed  user  attitudes  towards  the  problem-solving 
environment  and  checked  the  applied  manipulations  (feedback,  labeling  and  anonymity), 
(see  Appendix  A).  After  completion  of  all  experiment  tasks,  the  experiment  assistant 
debriefed  subjects  on  the  purpose  of  the  study. 

After  dismissing  the  peuticipants,  the  experiment  facilitator  saved  the  logs  from 
each  subjects  GSS  station  and  the  group  results  from  the  two  problem-solving  tasks  onto 
a  diskette  and  backed  up  the  GSS  session  to  a  diskette  and  to  the  hard  drive. 

3.7  Face-to-face  Tasks  and  Procedures 

All  steps  included  in  the  face-to-face  sessions  were  identical  to  the  GSS  sessions 
with  two  exceptions.  In  each  step  where  Group  Systems  software  was  used  in  the  GSS 
sessions,  a  pencil  and  paper  were  used  in  the  face-to-face  sessions.  Face-to-face  sessions 
were  also  provided  with  a  whiteboard,  markers  and  erasers  to  facilitate  their  discussions. 
Other  areas  that  were  replicated  in  the  face-to-face  sessions  to  mirror  the  GSS  sessions 
included  the  use  of  a  projection  screen,  scenario  item  formatting,  and  training.  The  use  of 
the  projection  screen  to  display  team  voting  results  in  the  GSS  sessions  was  incorporated 
into  the  face-to-face  sessions  to  ensure  all  participants  were  shown  team  voting  results  in 
the  same  manner.  During  the  group  vote,  the  items  voted  on  in  both  the  moon  and  desert 
scenarios  were  provided  in  the  same  format  as  the  GSS  groups,  again  ensuring  the 
starting  point  for  all  participants  was  identical.  Finally,  the  training  procedure  was 
modified  for  the  face-to-face  sessions  to  more  closely  resemble  the  computer  training 
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provided  to  GSS  participants  ensuring  that  extra  group  interaction  training  was  available 
to  all  participants. 

3.8  Hypothesis  Measures 

As  stated  in  Chapter  11,  this  study  is  based  upon  the  belief  that  manipulation  of 
anonymity  will  effect  user  participation,  group  decision  quality,  group  consensus,  user 
attitudes  and  user  satisfaction  with  the  group  outcome.  The  constructs  are  defined  in 
Table  9. 

The  first  construct,  user  participation,  measured  the  number  of  inputs  each  subject 
contributed  during  the  initial  15-minute  discussion  period. 

Group  Decision  Quality,  the  second  construct,  was  based  on  two  factors:  the  team 
score  on  the  moon  scenario  and  the  average  improvement  for  each  individual  when 
comparing  their  initial  solution  to  the  team’s  final  solution.  Average  improvement  was 
calculated  by  subtracting  the  team  score  from  each  individual  score  and  determining  the 
average  improvement  percentage. 

The  next  construct,  group  consensus,  was  measured  using  Ventana’s  Coefficient 
of  Concordance  (VCC),  which  is  based  on  Kendall’s  Coefficient  of  Concordance  (KCC). 
The  formula  used  to  calculate  consensus  is  as  follows: 

VCC  =  1 .00  -  ((STD/(RHV  -  RLV))  *  2) 

Where  STD  =  Standard  Deviation,  RHV  =  Highest  Possible  Allowed  Value,  and  RLV  = 
Lowest  Possible  Allowed  Value  (Ventana  GroupSystems  Workgroup  Edition  2.0 
Concepts  Guide,  1998). 
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To  ensure  consistency  in  measurements,  consensus  in  face-to-face  groups  was 
measured  by  inputting  the  face-to-group  votes  into  the  GSS  system  post-experiment  and 
letting  the  software  calculate  the  consensus  levels. 

Table  9. 

Construct  Definitions 

User  Participation 

Definition:  The  extent  to  which  participants  in  a  group  contribute. 

Group  Decision  Quality 

Definition:  Average  value  of  individual  decisions  reached  by  participants  in  a  group, 
measured  relative  to  an  optimal  decision  set.  (Lea,  1998) 

Group  Consensus 

Definition:  The  degree  of  agreement  generated  among  group  members  in  the  ranking  of 
the  group’s  main  ideas.  (Mejias  and  Shepherd,  1996:  140) 

User  Attitudes 

Definition:  Users’  dispositions  towards  the  performance  of  the  meeting  facilitator, 
utility,  usefulness,  and  perceived  ease  of  use  of  the  GSS,  and  dynamics  of  the  participant 
group.  (Lea,  1998) 

User  Satisfaction  with  the  Group  Outcome 

Definition:  The  extent  to  which  participants  in  a  group  are  committed  to  the  group 
decision. 
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A  benefit  of  using  the  NASA  Moon  Survival  Problem  and  the  Desert  Survival 
Problem  is  that  an  actual  consensus  number  can  be  obtained  by  comparing  the  group 
ranking  to  each  individuals  ranking.  This  comparison  results  in  a  set  of  deviation  scores 
identifying  the  level  of  agreement  between  the  group  ranking  and  each  individuals 
ranking  of  the  problems  fifteen  item  list.  Calculating  an  average  of  the  deviations  for 
each  groups  four  members  results  in  the  level  of  complete  agreement  or  consensus  among 
the  group  members.  The  lower  the  sum  of  the  deviations  the  higher  the  level  of 
consensus  and  the  higher  the  sum  of  the  deviations,  the  lower  the  level  of  consensus 
(Hirokawa,  1982). 

User  attitudes  employed  six  scales  that  measured  satisfaction  of  group  dynamics, 
perceived  ease  of  use,  perceived  utility,  perceived  ability  to  communicate,  perceived 
status  effects,  and  perceived  group  participation. 

The  final  construct,  user  satisfaction  with  the  group  outcome,  was  measured 
through  a  5-question  commitment  to  group  ranking  questionnaire  (see  Appendix  I). 

3.9  Survey  Design  and  Validation 

The  user  attitudes  construct  was  broken  down  into  six  measured  variables 
described  in  Table  10.  Each  variable  was  measured  using  a  seven-point  Likert  scale. 

This  scale  ranged  from  1  (Strongly  Disagree)  to  7  (Strongly  Agree)  and  4  (Neither  Agree 
Nor  Disagree)  in  the  middle.  Each  of  the  measured  variables  final  values  are  analyzed  on 
a  summative  scale  by  combining  the  results  of  five  questions  and  evaluating  the  mean. 
The  questionnaire  was  developed  by  combining  questions  for  user  satisfaction  with  the 
dynamics  of  the  participant  group,  user  belief  in  meeting  utility,  user  perception  of  ease 
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of  use,  and  user  participation  of  group  participation  used  by  Herberlie  and  Tolbert  (1999) 
with  user  perception  of  group  status  effects  used  by  Davison  (1997)  and  user  perception 
of  ability  to  communicate  used  by  Sperano  (1999).  These  questions  were  modified 

Table  10. 

Measured  Construct  Variable  Definitions 

User  Attitudes  _ 

User  Satisfaction  with  the  Dynamics  of  the  Participant  Group. 

Definition:  Degree  to  which  the  user  was  pleased  with  the  interactions  of  the  group. 

User  Perception  of  Ease  of  Use. 

Definition:  Degree  to  which  the  user  believes  that  using  the  Group  Support  System  will 
be  free  from  effort  (Davis,  1989). 

User  Belief  in  Meeting  Utility. 

Definition:  Degree  to  which  the  user  feels  the  meeting  format  (GSS  or  Face-to-face) 
provided  was  a  useful  aid  to  the  group  and  the  group  meeting  process. 

User  Perception  of  Ability  to  Communicate. 

Definition:  Degree  to  which  the  user  feels  they  were  able  to  share  information  with  and 
receive  information  fi'om  other  group  members. 

User  Perception  of  Group  Status  Effects. 

Definition:  Degree  to  which  the  user  believes 
User  Perception  of  Group  Participation. 

Definition:  Degree  to  which  the  user  feels  group  members  are  participating  and 
contributing  to  the  group. 
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slightly  to  more  fully  address  the  processes  used  in  this  study.  The  final  post-session 
questionnaire  included  thirty-nine  items  ([6  measured  variables  *  5  questions  each]  +  [3 
manipulation  checks  *  3  questions  each]  =  39  questions  total). 

Survey  data  was  first  analyzed  to  ensure  inter-item  reliability.  Data  was  entered 
into  an  EXCEL  spreadsheet  and  then  analyzed  using  SPSS  10.0  statistical  software.  The 
result  was  a  correlation  matrix,  reliability  coefficient,  mean,  and  standard  deviation  for 
each  set  of  questions  by  measured  variable.  Cronbach’s  coefficient  alpha  (a)  was  used  to 
estimate  the  internal  consistency  of  multi-item  scales.  A  reliability  analysis  for  each  of 
the  dependent  variables  which  constitute  the  user  attitudes  construct  are  presented  in 
Tables  11 -Table  16. 


Table  11. 


Reliability  Analysis  -  User  Satisfaction  with  the  Dynamics  of  the  Participant  Group 


M 

SD  a 

User  Satisfaction  with  the  Dynamics  of  the  Participant  Group 

6.02 

0.99  0.87 

I  would  not  mind  working  with  this  group  again. 

6.24 

0.88 

I  am  pleased  with  the  performance  of  our  group 

6.00 

0.89 

In  my  opinion,  we  worked  effectively  as  a  group. 

5.82 

1.12 

I  found  the  other  group  members  easy  to  work  with. 

6.03 

0.94 

I  enjoyed  participating  in  the  group  activity. 

6.04 

1.04 

62 


Table  12. 


Reliability  Analysis  -  User  Perception  of  Ease  of  Use 


M 

SD 

a 

User  Perception  of  Ease  of  Use 

6.07 

1.09 

0.90 

Learning  to  use  the  tools  and  process  provided  was  easy  for  me. 

6.52 

0.70 

I  found  it  easy  to  use  the  tools  and  process  to  share  information. 

6.07 

1.17 

I  found  it  easy  to  use  the  tools  and  process  to  receive  information. 

6.05 

1.12 

I  found  it  easy  to  use  the  tools  and  process  to  make  sense  of  shared 

5.89 

1.08 

information. 

I  found  it  easy  to  use  the  tools  and  process  to  help  my  group 

complete  the  task. 

5.83 

1.20 

Table  13. 

Reliability  Analysis  —  User  Belief  in  Meeting  Utility 

M 

SD 

a 

User  Belief  in  Meeting  Utility 

5.68 

1.25 

0.87 

The  tools  and  processes  helped  us  exchange  information. 

5.83 

1.16 

The  tools  and  processes  helped  us  make  good  use  of  the  information 

5.61 

1.22 

we  shared. 

The  tools  and  processes  helped  us  to  know  about  the  things  we 

5.74 

1.19 

agreed  on. 

The  tools  and  processes  helped  us  to  focus  on  the  points  where  we 

5.38 

1.39 

disagreed. 

The  tools  and  processes  helped  us  to  know  the  extent  we  achieved 

5.85 

1.23 

consensus. 
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Table  14. 


Reliability  Analysis  —  User  Perception  of  Ability  to  Communicate 


M 

SD 

a 

User  Perception  of  Ability  to  Communicate 

5.94 

1.18 

0.70 

I  experienced  few  problems  expressing  my  ideas  to  the  other  group 

5.50 

1.54 

members. 

I  felt  comfortable  putting  forward  my  own  ideas. 

6.39 

0.83 

I  had  little  trouble  understanding  the  points  made  by  other  group 

5.66 

1.24 

members. 

I  was  able  to  comment  on  the  ideas  submitted  by  other  group 

6.06 

0.99 

members  during  the  session. 

I  think  the  other  group  members  received  the  information  I  shared. 

6.07 

0.99 

Table  15. 

Reliability  Analysis  -  User  Perception  of  Group  Status  Effects 

M 

SD 

a 

User  Perception  of  Group  Status  Effects 

2.23 

1.48 

0.81 

One  or  more  of  the  group  members  tried  to  intimidate  the  others. 

2.10 

1.49 

One  or  more  of  the  group  members  tried  to  force  their  opinions  on 

the  group. 

2.74 

1.74 

I  felt  inhibited  from  participating  in  the  discussion  because  of  the 

1.67 

1.01 

behavior  of  one  or  more  of  the  other  members. 

I  felt  pressure  to  conform  to  a  particular  viewpoint. 

2.29 

1.43 

One  or  more  of  the  group  members  tried  to  dominate  the  discussion. 

2.32 

1.45 
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Table  16. 


Reliability  Analysis  -  User  Perception  of  Group  Participation 


M 

SD  a 

User  Perception  of  Group  Participation 

5.82 

1.07  0.82 

Everyone  in  the  group  was  very  involved  in  the  group’s  discussion. 

5.82 

1.13 

I  got  a  lot  of  good  ideas  about  ranking  from  the  other  members  of 

5.80 

1.04 

my  group. 

Everyone  in  my  group  seemed  to  contribute  all  of  the  ideas  they  had 

5.72 

1.08 

about  the  task. 

No  one  seemed  to  be  holding  back  information. 

5.82 

1.16 

My  group  shared  a  lot  of  information  while  we  completed  this  task. 

5.92 

0.91 

3.10  Data  Analysis 

The  hypotheses  identified  in  Chapter  n  tested  the  basic  belief  that  increasing  the 
level  of  anonymity  in  a  problem-solving  group  would  increase  decision  quality  and  user 
attitudes.  In  addition,  increasing  the  level  of  anonymity  would  decrease  the  level  of 
group  consensus  and  user  satisfaction  with  the  group  outcome.  Finally,  it  was  stated  that 
face-to-face  groups  (in  relation  to  all  GSS  treatments)  would  have  higher  user 
participation  and  that  increased  anonymity  in  GSS  treatments  would  result  in  higher 
levels  of  user  participation. 

The  data  was  analyzed  using  multivariate  ANOVA  (MANOVA)  procedure  in 
SPSS  10.0.  This  procedure  tests  for  differences  in  outcome  variables  among  the 
treatment  levels  while  controlling  for  the  overall  experiment-wise  error  rate.  The 
procedure  was  then  followed  with  the  univariate  ANOVA  procedure.  The  ANOVA 
procedures  compared  the  variance  within  the  sample  for  each  outcome  measure  that  can 
be  explained  by  the  treatment  conditions  to  the  total  variance  across  all  groups.  The  tests 
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of  the  null  hypothesis  for  the  effects  of  anonymity  on  each  measured  variable  were 
assessed  through  a  series  of  F-tests.  The  F-statistic  is  the  ratio  of  the  variabiUty  between 
treatment  groups  to  the  sampling  error  (McClave,  Benson,  and  Sincich,  1998).  The 
acceptable  statistical  reliability  threshold  was  set  at  a  <  0.05.  Each  univariate  ANOVA 
procedure  was  followed  by  a  set  of  pair  wise  comparisons  using  a  Bonferoni  procedure  to 
control  for  experiment  wise  error.  Therefore,  the  probability  of  a  Type  I  error  (falsely 
rejecting  a  null  hypothesis)  was  kept  to  less  then  five  percent.  The  variation  is  attributed 
to  the  difference  between  the  treatments  and  the  null  hypothesis  is  rejected  if  the  F- 
statistic  is  greater  than  the  critical  value  of  F  associated  with  a  <  0.05. 

The  ANOVA  table  provided  an  F-statistic  for  anonymity.  The  means  for  each 
measured  variable  were  plotted  according  to  the  four  levels  of  anonymity.  The  data  was 
analyzed  separately  for  unlabeled  groups  who  remained  completely  anonymous  or  chose 
to  label  themselves.  When  the  unlabeled  GSS  session  procedures  were  first  developed, 
verbiage  was  not  included  to  dissuade  subjects  from  labeling  themselves.  Subsequently, 
four  groups  (16  subjects)  chose  to  label  themselves.  At  that  point,  the  unlabeled  GSS 
procedures  were  modified  to  specifically  ask  the  subjects  “not  to  identify  themselves” 
once  any  subject  attempted  to  do  so.  This  development  did  not  impact  the  unlabeled  GSS 
session  sample  size  since  the  four  groups  that  chose  to  label  themselves  were  thrown  out 
and  the  sessions  were  rerun.  A  comparison  of  the  unlabeled  groups  that  remained 
anonymous  and  the  unlabeled  groups  that  chose  to  label  themselves  is  discussed  further 
in  Chapter  IV. 

The  index  eta-squared  (eta^)  was  used  to  assess  the  strength  of  relationships 
between  measured  variables.  Eta^  denotes  the  amount  of  variability  in  the  dependent 
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variable  that  can  be  attributed  to  the  independent  variable.  Eta^  values  can  range  from 
0.00  to  1.00.  As  eta^  approaches  1.00  a  stronger  relationship  between  variables  is 
signified,  however,  the  relationship  is  weaker  as  eta  approaches  0.00.  An  eta  less  then 
or  equal  to  0.05  is  considered  a  weak  effect,  0.10  a  moderate  effect,  and  greater  than  0.15 
a  strong  effect.  These  measures,  however,  must  be  considered  somewhat  arbitrary  and 
can  be  revised  downward  (Jaccard  and  Becker,  1997:275-276). 

3.11  Summary 

The  purpose  of  this  chapter  was  to  outline  the  steps  used  to  administer  an 
experiment  which  investigated  the  impact  of  different  levels  of  anonymity  on  user 
participation,  group  decision  quality,  group  consensus,  user  attitudes  towards  the 
problem-solving  environment,  and  user  satisfaction  with  the  group  outcome.  This 
chapter  described  the  methodology  employed  throughout  the  experiment,  along  with 
operationalized  constructs  and  variables  that  will  be  applied  to  measure  those  constructs. 
Lastly,  this  chapter  described  the  statistical  methods  used  to  analyze  the  data  collected. 

The  results  of  this  analysis  are  described  in  Chapter  IV.  The  interpretation  of  this 
analysis,  findings,  and  recommendations  for  future  research  are  presented  in  Chapter  V. 
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IV.  Analysis  of  Data 


4.1  Introduction 

This  chapter  presents  an  analysis  of  collected  data.  The  result  of  this  information 
in  terms  of  the  research  hypotheses  of  interest  to  this  work  is  described  in  Chapter  V 

A  multivariate  analysis  of  variance  (MANOVA)  was  used  to  determine  if 
significant  differences  on  measured  variables  existed  between  groups.  Results  of  this  test 
indicated  that  differences  between  treatment  types  did  exist  F(3,  268)  =  14.42,  p  <  .01, 
for  at  least  one  of  the  dependent  variables.  For  each  construct,  a  one-way  analysis  of 
variance  compared  the  means  of  decision  making  groups  using  unlabeled  GSS  session, 
labeled  without  placard  GSS  sessions,  labeled  with  placard  GSS  sessions,  and  face-to- 
face  sessions.  The  alpha  level  used  for  all  comparisons  was  .05.  In  addition,  a  multiple 
comparison  was  accomplished  using  the  Bonferroni  procedure  to  control  for  experiment- 
wise  error.  The  means  and  standard  deviations  of  all  the  constructs,  as  well  as  the  level 
of  significance  (if  any)  are  included  in  Table  17.  Where  significant  differences  were 
discovered  a  box  and  whiskers  diagram  is  included  for  clarity.  A  description  of  the  box- 
and-whisker  diagram  is  presented  in  Appendix  K.  A  discussion  of  the  findings  presented 
in  Table  17  are  further  discussed  in  the  following  sections. 

4.2  User  Participation  (HI) 

The  ANOVA  test  for  user  participation  was  found  to  be  statistically  significant, 
F(3, 268)  =  1 17.33,  /?  <  .01.  The  strength  of  the  relationship,  as  indexed  by  eta^  was  .57. 
The  results  of  this  test  indicated  that  the  mean  for  face-to-face  sessions  was  statistically 
(p  <  .01)  higher  than  the  mean  for  labeled  with  placard  GSS  sessions,  labeled  without 
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Table  17. 


ANOVA  Results 


Face-to- 

face 

Labeled 
GSS  with 
Placard 

Labeled 

GSS 

without 

Placard 

Unlabeled 

GSS 

Statistically  Reliable 
Differences 

M 

M 

M 

M 

iSD) 

(SD) 

(SD) 

(SD) 

HI:  User  Participation 

58.80 

22.00 

26.51 

22.95 

FTF>LP,LN,LN** 

(20.41) 

(8.50) 

(11.12) 

(9.74) 

H2:  Group  Decision  Quality 

Improvement  (%) 

61.25 

55.48 

47.92 

31.16 

FTF>NN** 

(63.8) 

(52.57) 

(42.98) 

(38.85) 

LP>NN* 

Group  Score 

21.15 

28.94 

28.61 

30.94 

(9.07) 

(7.23) 

(8.28) 

(6.06) 

H3:  Group  Consensus  (%) 

97.25 

89.61 

87.56 

89.94 

F1T>LP,LN** 

(2.77) 

(8.92) 

(6.89) 

(4.88) 

FTF>NN* 

H4:  User  Attitudes 

User  Satisfaction  with  Group 

6.26 

6.07 

5.85 

5.93 

FTF>LN* 

Dynamics 

(0.68) 

(0.71) 

(0.94) 

(0.77) 

User  Perception  of  Ease  of  Use 

6.10 

6.08 

6.06 

6.04 

(0.82) 

(0.90) 

(0.90) 

(0.99) 

User  Belief  in  Meeting  Utility 

6.08 

5.51 

5.46 

5.74 

FTF>LP,LN** 

(0.68) 

(1.17) 

(1.05) 

(0.94) 

Individual’s  Ability  to 

6.09 

5.93 

5.83 

5.91 

Communicate 

(0.65) 

(0.84) 

(0.83) 

(0.74) 

Status  Effects  of  the  Group 

2.26 

2.17 

2.24 

2.24 

(1.04) 

(1.03) 

(1.18) 

(1.15) 

Participation  in  Task  Related 

5.99 

5.80 

5.69 

5.81 

Areas 

(0.74) 

(0.86) 

(0.83) 

(0.80) 

H5:  User  Satisfaction  with 

5.89 

5.59 

5.35 

5.41 

FIP>LN,NN** 

Group  Outcome 

(0.83) 

(0.80) 

(0.83) 

(0.73) 

Note:  *  p  <  .05  **  p  <  .01 
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placard  GSS  sessions,  and  unlabeled  GSS  sessions.  There  was  no  statistically  significant 
difference  between  the  mean  for  labeled  with  placard  GSS  sessions,  labeled  without 
placard  GSS  sessions,  and  unlabeled  GSS  sessions.  The  dispersion  of  the  different 
treatment  user  participation  level  group  scores  is  displayed  in  Figure  9. 


FTF  L-P  L-N  N-N 

Level  of  Anonymity 

Figure  9.  Average  number  of  comments  submitted  per  group  member  by  treatment  type. 


4.3  Group  Decision  Quality  (H2) 

4.3.1  Improvement 

The  ANOVA  test  for  improvement  of  decision  making  groups  was  found  to  be 
statistically  significant,  F(3, 268)  =  4.15,p  <  .01.  The  strength  of  the  relationship,  as 
indexed  by  eta^,  was  .04.  The  results  of  this  test  indicated  that  the  mean  for  unlabeled 
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GSS  sessions  was  statistically  (p  <  .01)  lower  than  the  mean  for  face-to-face  sessions.  In 
addition  the  mean  for  unlabeled  GSS  sessions  was  statistically  ip  <  .05)  lower  than  the 
mean  for  labeled  with  placard  GSS  sessions.  Figure  4.2  highlights  the  caution  that  should 
be  used  when  interpreting  this  effect.  The  percentage  of  improvement  for  face-to-face  is 
skewed.  No  difference  is  indicated  in  the  median  values  between  face-to-face  and 
unlabeled  GSS  sessions.  There  was  no  statistically  significant  difference  between  the 
mean  for  face-to-face  sessions,  labeled  with  placard  GSS  sessions,  and  labeled  without 
placard  GSS  sessions.  The  dispersion  of  the  different  treatment  improvement 
percentages  is  displayed  in  Figure  10. 


160'- 

130' 


FTF  L-P  L-N  N-N 

Level  of  Anonymity 


Figure  10.  Average  improvement  of  group  score  when  compared  to  individual  scores  by 
treatment  type. 
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Of  the  68  groups,  44  produced  better  decisions  as  groups  after  discussion  than  the 
highest  score  of  any  individual  group  member  before  the  group  discussion.  When  broken 
down  by  treatment,  nine  of  the  groups  that  produced  a  poorer-quality  decision  than  their 
best  member  were  in  unlabeled  GSS  sessions.  The  other  fourteen  were  equally 
distributed  between  face-to-face  sessions  (5  groups),  labeled  with  placard  GSS  sessions 
(5  groups)  and  labeled  without  placard  GSS  sessions  (4  groups).  Finally,  only  one  group 
(an  unlabeled  GSS  session)  achieved  a  lower  group  score  then  the  average  of  the 
individuals  prior  to  the  discussion  period. 

4.3.2  Group  Score 

The  ANOVA  test  for  group  score  was  not  found  to  be  statistically  significant, 

F(3, 268)  =  1.94,  p  >  .05,  indicating  no  differences  between  the  mean  for  face-to-face 
sessions,  the  mean  for  labeled  with  placard  GSS  sessions,  the  mean  for  labeled  without 
placard  GSS  sessions,  and  the  mean  for  unlabeled  GSS  sessions. 

4.4  Group  Consensus  (H3) 

The  ANOVA  for  consensus  levels  was  found  to  be  statistically  significant,  F(3, 
64)  =  7.23,  p<  .01.  The  strength  of  the  relationship,  as  indexed  by  eta^,  was  .25.  The 
results  of  this  test  indicated  that  the  mean  for  face-to-face  sessions  was  statistically  (p  < 
.01)  higher  than  the  mean  for  labeled  with  placard  GSS  sessions  and  labeled  without 
placard  GSS  sessions.  In  addition,  the  mean  for  face-to-face  sessions  was  statistically  (p 
<  .05)  higher  than  the  mean  for  unlabeled  GSS  sessions.  There  was  no  statistically 
significant  difference  between  the  mean  for  labeled  with  placard  GSS  sessions,  labeled 
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without  placard  GSS  sessions,  and  unlabeled  GSS  sessions.  The  dispersion  of  the 
different  treatment  consensus  levels  is  displayed  in  Figure  1 1. 


N=  16  18  18  16 


FTP  L-P  L-N  N-N 

Level  of  Anonymity 

Figure  11.  Group  consensus  levels  by  treatment  type. 

4.5  User  Attitudes  (H4) 

4.5.1  User  Satisfaction  with  Group  Dynamics 

The  ANOVA  test  for  user  satisfaction  with  group  dynamics  of  decision  making 
groups  was  found  to  be  statistically  significant,  F(3, 268)  =  3.54,  p  <  .05.  The  strength  of 
the  relationship,  as  indexed  by  eta^,  was  .04.  The  results  of  this  test  indicated  that  the 
mean  for  face-to-face  sessions  was  statistically  ip  <  .05)  higher  than  the  mean  for  labeled 
without  placard  GSS  sessions.  There  was  no  statistically  significant  difference  between 
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labeled  without  placard  GSS  sessions  and  labeled  with  placard  GSS  sessions  or  unlabeled 
GSS  sessions.  The  dispersion  of  the  user  satisfaction  with  group  dynamics  group  scores 
is  displayed  in  Figure  12. 
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Figure  12.  User  satisfaction  with  group  dynamics  by  treatment  type. 


4.5.2  User  Perception  of  Ease  of  Use 

The  ANOVA  for  user  perception  of  ease  of  use  of  decision  making  groups  was 
not  found  to  be  statistically  significant,  F(3,  268)  =  0.05,  p  >  .05,  indicating  no 
differences  between  face-to-face  sessions,  labeled  with  placard  GSS  sessions,  labeled 
without  placard  GSS  sessions  or  unlabeled  GSS  sessions. 
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4.5.3  User  Belief  in  Meeting  Utility 

The  ANOVA  test  for  user  belief  in  meeting  utility  of  decision  making  groups  was 
found  to  be  statistically  significant,  F(3,  268)  =  5.43,  p  <  .01.  The  strength  of  the 
relationship,  as  indexed  by  eta^,  was  .06.  The  results  of  this  test  indicated  that  the  mean 
for  face-to-face  sessions  was  statistically  (p  <  .01)  higher  than  the  mean  for  labeled  with 
placard  GSS  sessions  and  the  mean  for  labeled  without  placard  GSS  sessions.  There  was 
no  statistically  significant  difference  between  labeled  with  placard  GSS  sessions,  labeled 
without  placard  GSS  sessions  or  unlabeled  GSS  sessions.  There  was  also  no  difference 
between  face-to-face  and  unlabeled  GSS.  The  dispersion  of  the  different  user  belief  in 
meeting  utility  group  scores  is  displayed  in  Figure  13. 


Level  of  Anonymity 

Figure  13.  User  belief  in  meeting  utility  by  treatment  type. 


75 


4.5.4  Individuals  Ability  to  Communicate 

The  ANOVA  test  for  individuals  ability  to  communicate  of  decision  making 
groups  was  not  found  to  be  statistically  significant,  F(3, 268)  =  1.37,  p  >  .05,  indicating 
no  differences  between  the  mean  for  face-to-face  sessions,  the  mean  for  labeled  with 
placard  GSS  sessions,  the  mean  for  labeled  without  placard  GSS  sessions,  and  the  mean 
for  unlabeled  GSS. 

4.5.5  Status  Effects  of  the  Group 

The  ANOVA  test  for  status  effects  of  the  group  members  was  not  found  to  be 
statistically  significant,  F(3, 268)  =  0.10,  p  >  .05,  indicating  no  differences  between  the 
mean  for  face-to-face  sessions,  the  mean  for  labeled  with  placard  GSS  sessions,  the  mean 
for  labeled  without  placard  GSS  sessions,  and  the  mean  for  unlabeled  GSS  sessions. 

4.5.6  Participation  in  Task  Related  Areas 

The  ANOVA  test  for  participation  in  task  related  areas  was  not  found  to  be 
statistically  significant,  F(3,  268)  =  \.54,p  >  .05,  indicating  no  differences  between  the 
mean  for  face-to-face  sessions,  the  mean  for  labeled  with  placard  GSS  sessions,  the  mean 
for  labeled  without  placard  GSS  sessions,  and  the  mean  for  unlabeled  GSS  sessions. 

4.6  User  Satisfaction  with  the  Group  Outcome  (H5) 

The  ANOVA  test  for  user  satisfaction  with  the  group  outcome  was  found  to  be 
statistically  significant,  F(3, 268)  =  5.95,  p<  .01.  The  strength  of  the  relationship,  as 
indexed  by  eta^,  was  .06.  The  results  of  this  test  indicated  that  the  mean  for  face-to-face 
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sessions  was  statistically  (p  <  .01)  higher  than  the  mean  for  labeled  without  placard  GSS 
sessions  and  unlabeled  GSS  sessions.  There  was  no  statistically  significant  difference 
between  the  mean  for  labeled  with  placard  GSS  sessions,  labeled  without  placard  GSS 
sessions,  and  unlabeled  GSS  sessions.  The  dispersion  of  the  different  treatment  user 
participation  level  group  scores  is  displayed  in  Figure  14. 
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Figure  14.  Average  level  of  user  commitment  to  group  decision  by  treatment  type. 

4.7  Comparison  of  unlabeled  groups  that  remained  anonymous  and  unlabeled  groups 
that  chose  to  label  themselves. 

As  mentioned  in  Chapter  HI,  four  unlabeled  GSS  groups  chose  to  identify 
themselves  during  the  GSS  session,  thus  circumventing  the  anonymity  manipulation 
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check.  The  data  for  these  participants  was  excluded  from  the  analysis  in  the  above 
sections,  but  further  analysis  between  truly  anonymous  GSS  groups  and  self-labeled  GSS 
groups  is  provided  in  Table  18  and  the  following  sections. 

Table  18. 


Summary  of  Means  between  unlabeled  and  self-labeled  GSS  groups 


Measure 

Self-labeled  GSS  groups 

Comments 

28.16 

33.75 

Improvement 

32% 

20% 

Group  Score 

30.94 

35.50 

Consensus 

91% 

83% 

User  Satisfaction  with  Group  Dynamics 

5.90 

5.96 

User  Perception  of  Ease  of  Use 

6.08 

5.80 

User  Belief  in  Meeting  Utility 

5.79 

5.50 

Individuals  Ability  to  Communicate 

5.88 

5.86 

Status  Effects  of  the  Group 

2.36 

2.36 

Participation  in  Task  Related  Areas 

5.81 

5.46 

User  Commitment  to  Group  Decision 

5.44 

5.54 

Anonymity  Manipulation  Check 

3.82 

5.56 

Comment  Labeling  Manipulation  Check 

3.45 

5.02 

Table  18  compares  the  results  of  the  experiment  measures  between  unlabeled 
GSS  groups  that  remained  anonymous  and  groups  that  chose  to  label  themselves.  Self- 
labeled  GSS  groups  provided,  on  average,  five  more  comments  per  participant  than 
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unlabeled  GSS  groups.  Group  decision  quality  was  lower  in  both  measures 
(improvement  and  group  score)  in  groups  that  chose  to  label  themselves.  In  addition, 
consensus  was  much  lower  in  self-labeled  GSS  groups.  As  for  the  user  attitudes  and  user 
satisfaction  with  group  outcome  constructs  there  were  minimal  differences  between  self- 
labeled  and  unlabeled  GSS  groups. 

4.7.1  Anonymity 

A  one-way  analysis  of  variance  compared  the  mean  anonymity  manipulation 
checks  of  decision  making  groups  using  unlabeled  GSS  sessions  and  self-labeled 
unlabeled  GSS  sessions.  This  test  was  found  to  be  statistically  significant,  F(l,  18)  = 
19.24,  p  <  .01.  The  strength  of  the  relationship,  as  indexed  by  eta^,  was  .52.  The  results 
of  this  test  indicated  that  the  mean  for  unlabeled  GSS  sessions  (Af  =  3.82,  SD  =  0.76) 
was  statistically  (p  <  .01)  less  than  the  mean  for  self-labeled  unlabeled  GSS  sessions  {M 
=  5.56, 5Z)  =  0.43). 


4.7.2  Comment  Labeling 

A  one-way  analysis  of  variance  compared  the  mean  comment  labeling 
manipulation  checks  of  decision  making  groups  using  unlabeled  GSS  sessions  and  self- 
labeled  unlabeled  GSS  sessions.  This  test  was  found  to  be  statistically  significant,  F(l, 
18)  =  10.88,p  <  .01.  The  strength  of  the  relationship,  as  indexed  by  eta^,  was  .38.  The 
results  of  this  test  indicated  that  the  mean  for  unlabeled  GSS  sessions  (M  =  3.45,  SD  = 
0.90)  was  statistically  (p  <  .01)  less  than  the  mean  for  self-labeled  unlabeled  GSS 
sessions  (M  =  5.02,  SD  =  0.58). 
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4.8  Summary 

This  chapter  presents  an  assessment  of  the  success  of  manipulations  included  in 
the  experimental  design  and  the  results  of  analysis  performed  on  data  collected  through 
survey  administration  and  direct  observation  of  experiment  participants.  The  results  of 
this  analysis  effort,  excluding  the  discussion  of  manipulation  success,  are  presented  in 
Table  19.  In  Chapter  V,  these  results  are  discussed  more  specifically  in  relation  to  the 
hypotheses  outlined  in  Chapter  II. 
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Table  19 


Summary  of  Effect  Size  and  Statistical  Significance 


Hypothesis 

Strength  of 
Effect  (Eta^) 

Signiffcant 

Differences 

HI 

User  Participation 

0.57  ** 

FrF>LP,LN,LN** 

H2 

Group  Decision  Quality 

Improvement  (%) 

0.04  ** 

FrF>NN** 

LP  >  NN  * 

Group  Score 

- 

- 

H3 

Group  Consensus 

0.25  ** 

FTP  >  LP,  LN  ** 

FTF>NN* 

H4 

U ser  Attitudes 

User  Satisfaction  with  Group  Dynamics 

0.04* 

FTF>LN* 

User  Perception  of  Ease  of  Use 

- 

- 

User  Belief  in  Meeting  Utility 

0.06  ** 

FTF  >  LP,  LN  ** 

Individual’s  Ability  to  Communicate 

- 

- 

Status  Effects  of  the  Group 

- 

- 

Participation  in  Task  Related  Areas 

- 

- 

H5 

User  Satisfaction  with  the  Group  Outcome 

0.06  ** 

FTF>LN,NN** 

Note:  *  p  <  .05  **p<.0\ 
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V.  Conclusions  and  Recommendations 

5.1  Introduction 

The  purpose  of  this  study  was  to  determine  the  impact  of  different  levels  of 
anonymity  on  decision-making  groups  in  an  effort  to  improve  user  participation  rates, 
group  performance,  group  consensus,  user  attitudes  and  user  satisfaction  with  the  group 
decision.  The  study  presented  a  conceptual  framework  in  Chapter  II  and  discussed 
numerous  studies  that  have  looked  at  content  and  process  anonymity.  In  order  to 
examine  the  hypothesis  presented  in  Chapter  II,  an  experiment  was  developed  that 
manipulated  anonymity.  The  main  effects  could  then  be  studied  to  determine  which  level 
of  anonymity  was  significantly  better  then  the  others  in  terms  of  user  participation,  group 
performance,  group  consensus,  user  attitudes,  and  user  satisfaction  with  the  group 
decision.  The  results  of  the  experiment  are  described  in  this  chapter  and  presented  in 
relation  to  the  hypothesis  from  Chapter  II.  Finally,  this  chapter  will  present  overall 
conclusions  from  this  study,  limitations  of  the  research  and  recommendations  for  future 
research. 

5.2  Hypothesis  1:  Effects  of  anonymity  on  user  participation 

Hypothesis  1  proposed  that  the  level  of  anonymity  provided  in  a  problem-solving 
environment  (GSS  only)  would  positively  impact  user  participation  levels,  but  that  face- 
to-face  groups  would  achieve  the  highest  participation  levels.  For  the  purpose  of 
evaluation,  the  four  levels  of  anonymity  were  broken  into  two  sub-hypotheses  that  are 
described  and  discussed  further. 
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5.2.1  Hypothesis  la 

Hypothesis  la  presented  the  idea  that  face-to-face  decision  making  groups 
would  achieve  the  highest  level  of  user  participation  of  all  treatment  groups.  Review 
of  data  analysis  presented  in  Chapter  IV  presents  strong  support  for  this  hypothesis. 
Face-to-face  groups  had  nearly  twice  as  many  comments  as  each  of  the  GSS 
treatment  groups.  This  supports  Hiltz  et  al’s  statement  introduced  in  Chapter  n  that 
“there  is  unquestionably  a  greater  amount  of  communication  flow  during  a  face-to- 
face  conference  than  during  a  computerized  conference  (1986:  236).”  These  results 
can  he  attributed  to  the  fact  that  people  can  speak  faster  than  they  can  type. 

5.2.2  Hypothesis  lb 

Hypothesis  lb  posited  that  higher  levels  of  anonymity  in  a  GSS  environment 
would  positively  impact  user  participation  levels.  The  results  of  ANOVA  performed 
on  data  collected  from  the  experiment  participants,  however,  presented  no  evidence 
to  support  this  hypothesis.  Participants  in  the  two  GSS  treatments  offering  the 
highest  anonymity  (Unlabeled  GSS)  and  the  least  anonymity  (Labeled  with  placard 
GSS)  contributed  nearly  equally.  This  comparison  is  trivial,  but  could  be  disproved 
or  identified  as  significant  if  the  power  was  increased.  The  higher  number  of 
comments  from  labeled  without  placard  GSS  groups  might  be  attributed  to  the  fact 
that  they  had  process,  but  not  content  anonymity.  Consequently,  any  entry  they 
provided  could  not  be  associated  with  them,  but,  since  they  were  able  to  identify 
their  own  comments,  it  allowed  them  to  socially  compare  their  level  of  performance 
with  others  in  the  group. 
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5.3  Hypothesis  2:  Effects  of  anonymity  on  group  decision  quality 

Hypothesis  2  proposed  that  the  level  of  anonymity  provided  in  a  problem-solving 
environment  (either  GSS  or  face-to-face)  would  positively  impact  group  decision  quality. 
Group  decision  quality  was  measured  in  two  ways:  group  improvement  when  compared 
to  individual  scores  and  group  score.  Evidence  provided  through  ANOVA  showed  no 
support  for  this  hypothesis  for  either  measurement. 

After  a  closer  look,  the  ANOVA  results  for  group  improvement  reported  exactly 
the  opposite  results  from  what  was  hypothesized.  Process  anonymity  in  this  study 
negatively  influenced  group  decision  quality.  This  contradicts  previous  research  that  the 
process  gains  associated  with  anonynuty  (reduced  evaluation  apprehension,  more  errors 
caught,  low-threat  environment)  outweigh  the  process  losses  (free  riding).  This  indicates 
there  must  be  other  significant  factors,  other  than  anonymity,  that  influence  group 
decision  quality. 

5.4  Hypothesis  3:  Effects  of  anonymity  on  group  consensus 

Hypothesis  3  proposed  that  the  level  of  anonymity  provided  in  a  problem-solving 
environment  (either  GSS  or  face-to-face)  would  negatively  impact  group  consensus.  The 
results  from  the  ANOVA  for  group  consensus  fail  to  support  the  above  hypothesis.  Face- 
to-face  groups  achieved  a  significantly  higher  level  of  consensus  then  all  three  GSS 
treatments,  but  there  was  no  significant  difference  between  the  GSS  treatments.  This 
difference  cannot  be  attributed  to  anonymity  because  there  was  no  difference  between  the 
GSS  groups.  This  finding  is  probably  attributable  to  other  differences  between  GSS  and 
face-to-face  meetings.  This  result  could  be  attributed  to  the  fact  that  computerized 
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conferences  tend  to  facilitate  opinion  giving  and  that  this  negatively  impacts  their  ability 
to  reach  consensus.  This  result  could  also  be  attributed  to  the  limitations  imposed  on  the 
GSS  problem  solving  groups.  The  GSS  has  tools  that  allow  participants  to  review 
comments  while  they  vote,  this  could  result  in  a  higher  consensus  levels.  However,  the 
experimental  design  used  did  not  allow  the  participants  to  use  this  GSS  capability,  thus 
limiting  the  effectiveness  of  the  GSS. 

5.5  Hypothesis  4:  Effects  of  anonymity  on  user  attitudes 

Hypothesis  4  proposed  that  the  level  of  anonymity  provided  in  a  problem-solving 
environment  (either  GSS  or  face-to-face)  would  negatively  impact  user  attitudes.  There 
was  no  evidence  from  the  ANOVA  results  that  supported  this  statement.  The  only  effects 
were  found  with  the  user  belief  in  meeting  utility  measurement  and  meeting  utility.  As 
with  consensus  and  participation,  the  differences  are  due  to  characteristics  of  face-to-face 
meetings.  The  contradictory  findings  of  the  studies  presented  in  Chapter  II  indicated  that 
the  level  of  anonymity  would  not  influence  user  attitudes.  The  lack  of  user  attitude 
findings  in  this  study  may  be  attributed  to  the  fact  that  great  effort  was  put  forth  to  make 
the  face-to-face  sessions  mirror  the  GSS  sessions,  thus  reducing  the  impact  on  user 
attitudes. 

5.6  Hypothesis  5:  Effects  of  anonymity  on  user  satisfaction  with  the  group  outcome 

Hypothesis  5  proposed  that  the  level  of  anonymity  provided  in  a  problem-solving 
environment  (either  GSS  or  face-to-face)  would  negatively  impact  user  satisfaction  with 
the  group  outcome.  ANOVA  results  showed  a  weak  positive  effect  for  face-to-face  when 
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compared  to  the  two  GSS  treatments  with  process  anonymity.  These  findings  support 
Rao  and  Monk’s  (1999)  belief  that  anonymous  participants  need  for  external  justification 
is  not  supported,  subsequently  their  level  of  commitment  to  the  group  decision  is  lower. 
Identified  participants  achieve  higher  levels  of  satisfaction  with  the  group  decision 
because  their  need  for  external  justification  is  satisfied. 

5.7  Conclusions  and  Recommendations 

Overall,  the  conclusion  of  this  study  suggests  that  process  anonymity  has  a  weak 
detrimental  effect  on  problem-solving  groups  in  terms  of  improvement  in  decision  quality 
and  satisfaction  with  the  group  outcome.  The  findings  outlined  in  Table  19  indicate 
moderate  to  strong  effects  on  user  participation  and  group  consensus  attributed  to 
differences  between  face-to-face  and  GSS.  Whether  these  differences  are  due  to  the 
quality  of  face-to-face  meetings  or  method  effects  cannot  be  determined.  For  instance, 
the  strong  effect  on  user  participation  might  be  due  to  the  method  of  generating 
comments  (i.e.  you  can  talk  faster  then  you  can  type  or  that  participants  dislike  periods  of 
silence  and  are  motivated  to  fill  voids  with  chatter).  The  moderate  effect  on  group 
consensus  may  be  attributed  to  limitations  imposed  on  GSS  participants  which  did  not 
allow  them  to  fully  use  the  GSS  capabilities,  thus  reducing  the  chances  of  achieving  high 
levels  of  consensus. 

Tables  2  and  3  presented  many  contradictory  findings  between  different  levels  of 
anonymity.  This  study  continued  the  trend  of  contradictory  findings  by  finding  mixed 
results  when  analyzing  anonymity  in  problem-solving  groups.  The  positive  and  negative 
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benefits  of  anonymity  presented  in  this  study  can  be  added  to  the  GSS  body  of 
knowledge. 

Despite  the  underwhelming  results,  this  study  is  important  for  its  attempt  to 
measure  the  independent  effects  of  process  and  content  anonymity.  The  introduction  of 
labeled  GSS  sessions  with  placards  went  one  step  beyond  the  present  research.  Including 
labeled  with  placard  GSS  sessions  in  future  GSS  research  efforts  could  prove  beneficial. 
Since  both  labeled  with  placard  GSS  sessions  and  face-to-face  sessions  provided  neither 
content  or  process  anonymity  the  lack  of  findings  on  three  of  the  five  constructs  when 
comparing  these  two  treatments  was  not  unexpected.  Further  analysis  between  these  two 
treatments  could  provide  more  detail  on  the  benefits  of  a  GSS  compared  to  a  face-to-face 
session.  This  analysis  would  reduce  the  focus  on  anonymity  and  direct  it  more  towards 
the  technology,  since  the  results  of  the  anonymity  manipulation  checks  found  nearly 
identical  results  between  face-to-face  groups  and  labeled  with  placard  GSS  groups. 

Finally,  the  findings  presented  in  this  study  should  emphasize  to  group  facilitators 
and  leaders  that  users  perceive  different  levels  of  anonymity  and  that  caution  should  be 
used  when  anonymity  is  manipulated  for  problem-solving  groups.  Consideration  should 
be  given  to  the  true  goal  of  the  meeting  (i.e.  consensus  or  decision  quality)  and  then  a 
level  of  anonymity  should  be  chosen  based  on  that  goal. 

5.8  Limitations  and  Recommendations  for  Future  Research 

The  underwhelming  results  considered  here  present  one  overarching  implication 
to  practitioners  and  academicians  alike:  the  results  from  any  one  study  do  not  apply  to  all 
group  work.  It  is  important  to  consider  all  the  limitations  before  applying  this  research  to 
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future  efforts.  Chapter  IV  highlighted  the  differences  present  between  the  different  levels 
of  anonymity.  This  studies  findings  and  noted  limitations  provide  a  platform  for  future 
anonymity  studies.  The  need  for  further  research  in  the  area  of  anonymity  and  problem¬ 
solving  groups  is  clear. 

A  limitation  of  this  study  was  the  length  of  the  experiment  (approximately  two 
hours).  Since  four  studies  were  consolidated  into  one  experiment  the  study  had  to 
incorporate  multiple  questionnaires  and  manipulations  that  could  influence  the 
participants  perceptions  of  the  anonymity  manipulations.  The  post-questionnaire,  which 
measured  user  attitudes,  was  the  last  step  of  the  experiment  and  possibly  could  have  been 
impacted  by  the  experiment  length  because  of  task  saturation.  On  one  occasion,  the 
length  of  the  experiment  clearly  affected  a  participant.  The  participant  was  so  rushed  to 
get  to  his  next  activity  that  on  the  post-questionnaire,  they  rushed  through  and  marked  all 
Likert-scale  items  with  a  “4”.  The  data  point  was  not  used  but  indicated  the  possibility 
that  the  experiment  length  could  have  been  too  long. 

A  second  limitation  of  the  study  was  group  size.  This  study  used  a  group  size  of 
four,  which  is  common  in  many  GSS  experiments.  However,  a  larger  group  may  be 
necessary  to  more  accurately  measure  the  effects  of  anonymity  on  problem-solving 
groups.  In  smaller  groups,  it  may  still  be  possible  for  group  members  to  identify  the 
source  of  conunents  even  if  they  are  unlabeled  because  of  certain  terms  an  author  is 
known  to  use.  If  the  group  were  larger,  this  likelihood  would  be  reduced.  Larger  groups 
would  likely  enhance  the  benefits  and  drawbacks  of  anonymous  problem  solving  groups. 
With  a  larger  group,  more  conunents  would  likely  be  generated,  but  this  could  also  result 
in  a  participant’s  tendency  to  free  ride.  In  a  face-to-face  group,  only  one  person  can 
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speak  at  a  time.  Subsequently,  in  a  fixed  period,  there  is  a  maximum  limit  on  the  number 
of  comments  generated.  This  is  not  the  case  in  a  GSS  session.  Logically  there  should  be 
a  group  size  in  which  the  GSS  can  provide  more  comments  than  a  face-to-face  group. 
This  would  likely  require  a  task  that  can  generate  many  comments.  Once  you  exceed  the 
minimum  group  size,  the  question  then  becomes  whether  people  can  or  will  pay  attention 
to  the  additional  information.  One  final  concern  regarding  group  size  is  real  world  group 
support  systems  often  have  many  more  participants.  Analyzing  the  impact  of  anonymity 
in  different  sized  groups  is  another  direction  future  research  could  take. 

A  third  limitation  of  the  study  was  that  the  task  used  was  strictly  an  experimental 
task.  In  this  study,  participants  had  no  strong  personal  involvement  in  completing  the 
task  and  were  not  challenged  emotionally.  Using  a  more  emotionally  charged  task  or  one 
that  facilitated  a  higher  level  of  personal  involvement  likely  would  have  shown  different 
effects  attributable  to  the  anonymity  manipulation.  Thus,  future  research  might  be 
usefully  directed  at  studying  the  impact  of  anonymity  when  accomplishing  an 
emotionally  loaded  task. 

A  final  limitation  of  the  study  was  that  the  majority  of  the  participant  groups 
already  knew  each  other  before  the  experiment.  In  some  cases,  participants  had  know 
each  other  for  only  a  short  time  (one  week),  while  in  other  cases,  participants  had  know 
each  other  for  as  long  as  three  years.  This  level  of  familiarity  may  have  an  interactive 
effect  with  anonymity  on  the  results  and  was  not  measured.  Future  studies  should  look  at 
the  impact  of  anonymity  when  groups  are  familiar/unfamiliar  with  each  other. 
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5.9  Summary 

Anonymity  has  been  one  of  the  most-studied  GSS  components.  The  results 
provided  in  this  studies  literature  review  and  findings  are  mixed  indicating  the  need  for 
further  research  in  the  area  of  anonymity  and  problem-solving  groups. 

As  Nunamaker,  Dennis,  Valacich,  Vogel  and  George  (1993)  point  out  “anonymity 
is  not  always  appreciated  or  warranted  (144).”  Finding  the  right  situation  and 
environment  in  which  anonymity  is  appreciated  and  warranted  is  still  a  challenge  facing 
researchers  and  practitioners  alike.  Finding  the  correct  balance  between  freely 
contributing  without  fear  and  the  desire  for  recognition  remains  a  significant  challenge. 
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Appendix  A:  Post-Test  Questionnaire 


Answer  the  questions  using  the  following  scale 

1.  Strongly  Disagree 

2.  Disagree 

3.  Disagree  Somewhat 

4.  Neither  Agree  Nor  Disagree 

5.  Agree  Somewhat 

6.  Agree 

7.  Strongly  Agree 

_ I  would  not  mind  working  with  this  group  again. 

_ I  am  pleased  with  the  performance  of  our  group. 

_  In  my  opinion,  we  worked  effectively  as  a  group. 

_  I  found  the  other  group  members  easy  to  work  with. 

_  I  enjoyed  participating  in  the  group  activity. 

_ _  Learning  to  use  the  tools  and  process  provided  was  easy  for  me. 

_  I  found  it  easy  to  use  the  tools  and  process  to  share  information. 

_  I  found  it  easy  to  use  the  tools  and  process  to  receive  information. 

_ I  found  it  easy  to  use  the  tools  and  process  to  make  sense  of  shared  information. 

_  I  found  it  easy  to  use  the  tools  and  process  to  help  my  group  complete  the  task. 

_ The  tools  and  processes  helped  us  exchange  information. 

_  The  tools  and  processes  helped  us  make  good  use  of  the  information  we  shared. 

_  The  tools  and  processes  helped  us  to  know  about  the  things  we  agreed  on. 

_ The  tools  and  processes  helped  us  to  focus  on  the  points  where  we  disagreed. 

_  The  tools  and  processes  helped  us  to  know  the  extent  we  achieved  consensus. 
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Answer  the  questions  using  the  following  scale 


1.  Strongly  Disagree 

2.  Disagree 

3.  Disagree  Somewhat 

4.  Neither  Agree  Nor  Disagree 

5.  Agree  Somewhat 

6.  Agree 

7.  Strongly  Agree 

_ I  experienced  few  problems  expressing  my  ideas  to  the  other  group  members. 

_  I  felt  comfortable  putting  forward  my  own  ideas. 

_  I  had  little  trouble  understanding  the  points  made  by  other  group  members. 

_  I  was  able  to  comment  on  the  ideas  submitted  by  other  group  members  during  the 

session. 

_  I  think  the  other  group  members  received  the  information  I  shared. 

_  One  or  more  of  the  group  members  tried  to  intimidate  the  others. 

_  One  or  more  of  the  group  members  tried  to  force  their  opinions  on  the  group. 

_  I  felt  inhibited  from  participating  in  the  discussion  because  of  the  behavior  of  one 

or  more  of  the  other  members. 

_  I  felt  pressure  to  conform  to  a  particular  viewpoint. 

_  One  or  more  of  the  group  members  tried  to  dominate  the  discussion. 

_ Everyone  in  the  group  was  very  involved  in  the  group’s  discussion. 

_ I  got  a  lot  of  good  ideas  about  ranking  from  the  other  members  of  my  group. 

_ Everyone  in  my  group  seemed  to  contribute  all  of  the  ideas  they  had  about  the 

task. 

_  No  one  seemed  to  be  holding  back  information. 
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Answer  the  questions  using  the  following  scale 

1.  Strongly  Disagree 

2.  Disagree 

3.  Disagree  Somewhat 

4.  Neither  Agree  Nor  Disagree 

5.  Agree  Somewhat 

6.  Agree 

7.  Strongly  Agree 


My  group  shared  a  lot  of  information  while  we  completed  this  task. 

My  group  received  information  on  how  well  we  shared  information  during  the 
first  task. 

Each  member  of  my  group  knew  how  much  they  had  contributed  to  the  group 
during  the  first  task. 

I  knew  how  much  information  other  members  of  my  group  shared  during  the  first 
task. 

I  could  recognize  the  originator  of  most  comments. 

Other  group  members  could  connect  me  to  the  comments  I  made. 

Other  group  members  knew  when  I  made  a  contribution  to  the  group. 

I  could  tell  if  someone  was  sharing  more  information  than  other  members  of  the 
group. 

I  could  tell  if  someone  participated  less  than  other  members  of  the  group. 

Other  group  members  could  judge  the  extent  that  I  participated  in  the  group. 
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Appendix  D:  GSS  Experiment  Script 


GSS:  Pre-Experiment  Steps 

Ensure  following  items  are  available: 

Big  folder  labeled  consent  forms 
4  Manila  folders 
Attached  via  paper  clip  are 
Consent  form 

Demographic/Personality  Questionnaire 

1  copy  of  Moon  Scenario 

Check  out  Projector  and  printer  with  paper 

In  Group  System  Admin,  click  on  Clear,  then  open  roster,  edit  user  terminal,  set  to  full-access 
user 


Start  Group  Systems  WGE  at  Facilitator  station  and  all  user  stations 
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5.  Ensure  logs  are  clear  on  each  subject’s  machine 


Copy  all  activities  for  session  from: 

For  a  labeled  session:  GSS  Labeled  Study  —  clean  copy 
For  an  unlabeled  session:  GSS  Unlabeled  Study  -  clean  copy 

Paste  to  GSS  Study  -  Current 
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Configure  each  GSS  station  for  EACH  ACTIVITY  on  facilitator  station 


Group  SeUings 


Categorizer 


Researchers  Label  Subjects  Monitors  with  placard  (if  applicable) 
Ensure  four  placards  (blue,  green,  red  and  yellow)  are  available 
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Prep  Room:  Introduction 

When  subjects  arrive,  introduce  yourself.  Have  subjects  wait  in  the  prep  room.  Tell  subjects 
“The  task  will  begin  when  all  participants  have  arrived.  ” 

Once  all  participants  have  arrived,  have  all  subjects  sit  down  in  the  prep  room. 

Facilitator  says:  “Welcome  to  the  study.  I’m  XK  and  this  is  XX.  We  are  AFIT  students 
conducting  an  experiment  for  our  Masters  degree.  We  will  be  asking  you  some  questions  about 
yourself.  Our  study  looks  at  how  different  types  of  groups  interact  to  solve  a  problem.  During 
the  course  of  this  experiment  you  will  be  asked  to  complete  three  questionnaires,  receive  some 
group  interaction  training,  and  conduct  tasks  individually  and  as  a  group.  About  halfway 
through  this  two  hour  experiment  you  will  be  given  a  short  break.  ” 

Facilitator  says:  “My  assistant  will  now  hand  out  a  manila  folder  with  some  attached 
information.  Please  don’t  look  at  the  attached  information  until  asked.  ” 

Assistant  provides  participants  with  manila  folder. 

Facilitator  says:  “To  begin,  please  remove  the  consent-form  from  the  manila  folder.  This  form 
indicates  your  rights  as  a  participant  in  the  study.  Please  read  the  consent  form  and  print  and 
sign  your  name  at  the  bottom  of  the  page.  Your  participation  is  voluntary.  If  at  any  time  you 
want  to  stop  please  let  the  facilitator  know.  ” 

Subjects  read  and  sign  (if  applicable)  Consent  Form 

Assistant  collects  consent  forms 

Facilitator  says,  “This  is  the  only  place  your  name  will  be  recorded  during  this  experiment.  ” 
Assistant  puts  consent  forms  in  big  folder  labeled  consent  forms. 

Facilitator  says:  “We  would  now  like  you  to  fill  out  the  individual  characteristics  questionnaire 
attached  to  the  manila  folder.  All  responses  to  this  questionnaire  are  completely  confidential 
and  will  not  be  associated  with  you  as  an  individual  Use  the  rating  scale  provided  to  indicate 
how  accurately  each  statement  describes  you.  Think  about  yourself  as  you  generally  are  now 
and  not  as  you  wish  to  be  in  the  future.  Please  read  each  statement  carefully.  Does  anyone  have 
any  questions?” 

Subjects  complete  questionnaire. 

Facilitator  says:  “Please  place  the  questionnaire  in  the  manila  folder.  ” 

Facilitator  says:  “Now  lets  complete  a  problem  solving  task  individually.  Please  read  the 
scenario  and  complete  the  exercise.  It  will  take  you  approximately  5  minutes  to  complete  the 
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exercise.  If  you  finish  early,  please  remain  quiet  until  everyone  completes  the  exercise.  Please 
remove  the  scenario  from  your  manila  folder.  ” 

Facilitator  says:  “Please  begin.  ” 

Subjects  individually  complete  Moon  Scenario 

Facilitator  says:  “Please  place  the  Moon  Scenario  in  the  manila  folder.  ” 

Facilitator  says:  “Before  we  move  to  another  room  let’s  discuss  group  decision  making  and 
problem  solving  in  general.  The  first  step  is  for  the  group  to  discuss  the  problem  and  all 
pertinent  issues  related  to  the  problem.  One  method  often  used  to  do  this  is  “brainstorming” 
during  which  ideas  are  freely  generated  and  not  judged  on  quality  or  feasibility.  Once  the 
brainstorming  session  is  complete,  the  group  then  attempts  to  reach  consensus  on  a  solution. 
This  does  not  necessarily  mean  all  individuals  completely  agree  with  the  groups’  decision,  but 
the  decision  is  one  that  all  can  endorse.  There  are  different  methods  groups  use  to  reach 
consensus,  one  of  which  is  voting.  If  the  results  of  the  group  vote  indicate  agreement,  then 
consensus  is  reached.  If  the  group  does  not  have  agreement,  further  discussion  may  be  required 
to  reach  consensus.  Remember  the  purpose  of  this  study  is  to  look  at  how  different  types  of 
groups  interact  to  solve  a  problem.  Does  anyone  have  any  questions  before  we  move  to  the  next 
phase  of  the  study.  ” 

Facilitator  says:  “Please  pick  up  your  manila  folder  and  follow  me.  ” 

Assistant  moves  subjects  to  Task  Room  (GSS  Room) 
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GSS  Room:  Training 

Assistant  says:  “Please  take  a  seat  at  one  of  the  computers.  ” 

Subjects  sit  at  one  of  the  GSS  stations 
Training  Script 
Facilitator  flips  UP  projector 

As  you  introduce  options  in  GSS  point  to  them  on  the  screen. 

Facilitator  says:  “A  group  support  system  is  made  up  of  software,  computers  and  a  facilitator. 
Each  of  your  computers  has  Group  System  software  (point  to  screen)  loaded  on  it.  This  software 
and  hardware  is  often  used  in  the  Air  Force  to  increase  the  effectiveness  of  decision-making 
groups.  ” 

Facilitator  says:  “We  will  only  be  introducing  you  to  a  small  set  of  the  capabilities  of  a  GSS 
because  of  our  limited  time.  As  you  use  this  software,  please  only  use  the  capabilities  we 
introduce  to  you  so  we  can  minimize  the  impact  on  your  time  and  ours.  For  the  purposes  of  this 
study  we  will  be  using  two  GSS  tools:  Categorizer  and  Vote.  ” 

PAUSE 

Facilitator  says:  “Before  we  begin  the  actual  group  problem-solving  scenario,  we  will  first  guide 
you  through  a  brief  training  session.  You  will  be  introduced  and  allowed  to  practice  with  GSS 
Categorizer  and  Vote  tools.  Let’s  begin.  ” 
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Facilitator  starts  participants  in  Training  (Categorizer) 


I  GioupSystems  -  GSS  Study  --  Cunenl  -  [Training  (Calegotizer)) 


Facilitator  says:  “At  this  point,  you  should  have  been  invited  to  join  an  activity.  Please  click  on 

Yes.  ”  (Note  to  Researcher:  In  some  cases  this  field  will  be  blank.) 


Facilitator  says:  “You  may  receive  another  log-in  prompt.  Please  click  on  OK.  ” 

Facilitator  says:  “You  should  now  see  a  list  of  six  names  and  a  category  called  “Group 
Discussion”.  Double  Click  on  Group  Discussion.” 


Facilitator  says:  “A  new  window  should  appear  on  your  screen.  This  is  a  discussion  area  where 
you  will  provide  comments  for  the  group  problem-solving  task.  At  this  point  your  cursor  should 
be  in  the  large  field  at  the  bottom  of  the  window.  This  is  the  box  where  you  enter  your 
comments.  Please  type  in  one  method  you  would  use  to  rank  order  the  list  of  names.  ” 

PAUSE 


Facilitator  says:  “Click  on  the  Submit  key  at  the  bottom  of  the  window  on  the  left.  The  comment 
you  entered  should  appear  in  the  notepad  above  the  large  field.  Everyone  in  the  GSS  session 
will  be  able  to  see  all  comments  submitted.  Does  anyone  not  see  other’s  comments?” 

For  a  Labeled  Session  the  facilitator  says:  “If  you  look  at  the  end  of  each  comment  you  will  see 
that  the  GSS  software  labels  the  person  who  entered  the  comment.  You  should  see  our  choice  of 
labels  (blue,  green,  red  or  yellow)  at  the  end  of  each  comment.  ” 
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Facilitator  says:  “As  your  group  brainstorms  and  you  enter  your  individual  comments,  all  of  you 
will  be  able  to  see  the  inputs  of  the  entire  group.  Reading  others’  thoughts  and  ideas  allows  you 
to  “piggyback”  off  each  other  which  should  improve  your  group  brainstorming  process.  ” 

Facilitator  says:  “Now  that  we’ve  shown  you  how  to  enter  comments,  we  will  now  have  you 
perform  a  practice  session  before  we  move  into  the  problem-solving  task.  Your  group’s  task  is  to 
discuss  possible  ways  your  group  could  rank  order  the  names.  Any  and  all  comments  are 
valuable,  including  ideas  on  how  to  rank  the  names,  and  your  thoughts/opinions  of  each  other’s 
ideas.  You  will  have  a  couple  of  minutes  to  discuss  the  task  as  a  group.  At  the  end  of  the  session 
we  will  measure  group  consensus  on  how  you  ranked  the  list  by  introducing  you  to  the  GSS  Vote 
tool.  ” 

Facilitator  says:  “Please  begin  discussing  the  task.  ” 

Assistant  notifies  facilitator  when  time  reaches  5  minutes. 

Facilitator  says:  “Please  stop  discussing  the  task  at  this  time.” 

Facilitator  says:  “You’ve  had  plenty  of  time  to  discuss  possible  ways  to  rank  order  the  list  of 
names.  Now  it’s  time  to  actually  rank  the  names.  Hopefully,  during  the  discussion  period,  your 
group  decided  how  to  rank  the  list.  We  will  now  introduce  you  to  the  GSS  Vote  tool  where  each 
of  you  will  individually  rank  the  list  of  names.  Please  close  the  Group  Discussion  window.  ” 

Facilitator  closes  training  categorizer  and  selects  voting  method  for  ballot  and  clicks  OK. 


Facilitator:  Start  participants  in  Training  (Vote). 
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Facilitator  says:  “You  have  been  invited  to  join  the  Voting  activity.  Please  click  yes.  Now  you 
should  see  the  original  list  of  names.  You  change  the  sequence  of  the  list  by  clicking  and 
dragging  an  item  to  the  position  in  the  list  you  wish  to  move  it.  Please  begin  voting  by  re¬ 
ordering  the  list  now.  ” 

PAUSE 

Facilitator  says:  “Once  you  are  satisfied  with  your  list  order,  cast  your  ballot  by  clicking  on  the 
“cast  ballot”  icon,  which  is  the  2'^ from  the  left.  You  will  receive  a  dialogue  box  asking  you  to 
confirm  your  ballot.  Please  click  yes  and  wait  for  further  instructions. 
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Ensure  voting  graph  tool  is  set  as  follows 


Facilitator:  Open  the  result  window  (bargraph)  to  monitor  individual  votes. 


^  GroupSystems  -  GSS  Study  Current  -  [Moon  (Vole)] 
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When  n=number  of  participants,  the  group  is  done.  Display  the  results  with  the  projector  and 
explain  the  level  of  group  consensus.  _ _ 


Facilitator  says:  “During  the  actual  problem-solving  tasks  following  this  training,  your  group 
will  be  given  5  minutes  after  the  initial  vote  to  allow  your  group  to  determine  if  everyone  is 
satisfied  with  the  final  solution,  or  if  further  discussion  is  needed.  ” 


Training  exercise  complete. 

Facilitator  says:  “I  will  now  be  closing  the  training  session  and  beginning  the  first  exercise. 
Please  do  not  enter  any  information  until  instructed.  ” 


Facilitator  stops  participants  in  Training  (Categorizer). 
Facilitator  stops  participants  in  Training  (Vote). 


Facilitator  flips  DOWN  projector 
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GSS  Room:  Experiment  One 

Moon  Scenario  Script 

Facilitator  says:  “You  will  have  15  minutes  to  discuss  the  scenario  as  a  group.  The  scenario  you 
will  be  discussing  is  the  same  one  you  did  previously  as  individuals.  At  the  end  of  the  15  minutes 
you  will  each  rank  order  the  list  individually.  The  ranking  results  will  be  consolidated  as  a 
group  to  indicate  how  well  the  group  reached  consensus.  At  this  time  the  group  will  have 
another  5  minutes  to  determine  if  you  have  reached  consensus.  If  the  group  did  not  reach 
consensus  you  should  try  to  resolve  any  differences.  The  group  will  then  individually  rank  the 
items  again.  ” 

Facilitator  says:  “During  the  15-minute  discussion  period,  focus  on  discussing  each  item’s 
merits,  not  on  where  each  item  should  be  ranked.  You  will  be  able  to  rank  order  the  list  at  the 
end  of  the  15  minutes.  It  is  important  to  focus  on  discussion  not  on  how  to  rank  order  since  this 
could  shut  down  conversation.  It  is  normal  to  experience  a  lull  during  conversation,  but  this  is 
part  of  group  dynamics.  ” 

Facilitator  says:  “Researchers  will  not  answer  questions  dealing  with  the  scenario  during  this 
session,  but  will  assist  with  GSS  questions.  We  will  let  you  know  when  there  are  5  and  2  minutes 
left  in  the  session.  ” 

Facilitators  start  participants  in  Moon  Scenario  --  Categorizer. 

Facilitator  says:  “At  this  point,  you  should  have  been  invited  to  join  an  activity.  Please  click  on 

Yes.  ”  (Note  to  Researcher:  In  some  cases  this  field  will  be  blank.) 

Facilitator  says:  “Now  you  should  be  prompted  to  enter  an  author  tag.  Please  click  on  OK.  ” 

Facilitator  says:  “You  have  a  list  of  15  items  and  a  category  called  “Group  Discussion” .  Double 
Click  on  Group  Discussion.” 

Facilitator  says:  “Are  there  any  questions?” 

Facilitator  says:  “You  may  begin  group  discussion  on  the  Moon  Scenario.  ” 


10:00 

FACILITATOR  INPUT:  YOU  HAVE  5  MINUTES  LEFT  TO  DISCUSS 
THE  SCENARIO  IF  NEEDED. 

13:00 

FACILITATOR  INPUT:  YOU  HAVE  2  MINUTES  LEFT  TO  DISCUSS 
THE  SCENARIO  IF  NEEDED. 

Subjects  finish  initial  discussion. 

Facilitator  says:  “Please  close  your  Group  Discussion  Window.  ” 
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Facilitator  says:  “We  will  now  open  a  voting  tool  for  your  use.  ” 

Facilitator  starts  Moon  (Vote). 

Facilitator  says:  “You  have  been  invited  to  join  the  Voting  activity.  Please  click  yes.  Now  you 
should  see  the  original  list  of  items.  Please  begin  voting  by  re-ordering  the  list  now  just  as  you 
did  in  the  training  session.  ” 

PAUSE  (1  minute) 

Facilitator  says:  “Once  you  are  satisfied  with  your  list  order,  cast  your  ballot  by  clicking  on  the 
“cast  ballot”  icon,  which  is  the  from  the  left.  You  will  receive  a  dialogue  box  asking  you  to 
confirm  your  ballot.  Please  click  yes  and  wait  for  further  instructions.  ” 

Ensure  voting  graph  tool  is  set  as  follows: 


Facilitator:  Open  the  result  window  to  monitor  individual  votes.  When  n=number  of 
participants,  the  group  is  done.  Display  the  results  with  the  projector  and  explain  the  level  of 
group  consensus. 


Ill 


Facilitator  says:  “You  have  the  next  five  minutes  to  discuss  the  results  further  using  the  GSS.  If 
you  are  satisfied  with  the  results  let  the  facilitator  know.  If  not  you  will  be  given  the  chance  to 
vote  again  at  the  end  of  the  five  minutes.  ” 


ff  group  is  satisfied  with  original  outcome  then  go  to  STEP  2  else  do  the  following: 

Facilitator  says:  “Go  ahead  and  vote  again.  ” 

Facilitator:  Open  the  result  window  to  monitor  individual  votes.  Assistant  lets  Facilitator  know 
when  the  group  is  done.  Display  the  results  with  the  projector  and  explain  the  level  of  group 
consensus. 

Facilitator  says:  “These  are  your  final  results.  “ 


Facilitator  stops  Moon  (Vote). 

Facilitator  flips  DOWN  projector 

Facilitator  says:  “Feel  free  to  take  a  quick  5-minute  break  in  the  prep  room.  Please  don’t  discuss 
what  color  you  are.  ” 


Assistant  counts  number  of  comments  per  subject  and  creates  appropriate  feedback  and  goal 
charts. 
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Facilitator  stops  Subjects  in  Moon  Scenario  —  Categorizer 
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Prep  Room:  Feedback 

Subjects  come  back  from  break 

Assistant  says:  “We  would  now  like  to  give  you  a  short  questionnaire  concerning  your  groups 
ranking  on  the  task  you  just  completed.  ” 

Assistant  provides  subjects  with  5  item  commitment  to  ranking  Questionnaire 
Goal  and  No  Feedback  or  Goal  Feedback  Script 


Assistant  says:  ‘‘During  the  group  exercise  Just  completed,  your  group  worked  together  to 
solve  a  problem.  Studies  have  shown  that  when  individual  members  of  the  group  participate 
fairly  equally,  the  meeting  will  produce  better  results.  For  example,  as  you  can  see  in  the 
graph  (show  graph  of  equal  proportion)  the  participation  rates  were  almost  equal  among  the 
group  participants.  The  next  graph  shows  participation  rates  where  participants  did  not 
participate  equally.  What  problem  can  result  from  the  unequal  levels  of  participation?” 

(Wait  for  group  to  respond...  Look  for  an  answer  such  as  subject  4  did  not  participate  as 
much  and  he  may  have  had  the  best  answer  while  subject  2  dominated  the  meeting  with  his 
ideas.  If  group  does  not  submit  the  answer  looked  for,  provide  an  explanation. 

Assistant  says:  “Studies  have  also  shown  that  the  more  comments  input  during  a  meeting,  the 
greater  the  chances  to  reach  a  high  quality  decision.  In  other  words,  the  more  ideas  that  are 
generated  the  better  the  chance  the  optimum  solution  will  be  found  in  those  comments.  ” 

Assistant  says:  GOAL  STATEMENT:  “In  the  next  task,  try  to  participate  equally  while 
maximizing  your  number  of  comments.  ” 

IF  GIVING  FEEDBACK  READ  THE  NEXT  SECTION,  IF  NOT  STOP  AND  PROCEED  TO 
NEXT  TASK 

Assistant  says:  “I  will  now  show  each  of  you  a  graph  showing  your  participation  level  in  the 
previous  task.  “ 

Assistant  provides  an  explanation  of  the  feedback. 

Assistant  says:  “Next  to  the  bar  graph  of  each  individual  show  their  score.  Once  all  subjects 
have  looked  at  the  paper  give  them  the  paper  to  start  the  desert  scenario.  ” 

Assistant  says:  “You  will  now  be  given  5  minutes  to  complete  the  desert  scenario.  Please  follow 
the  directions  on  the  page.  ” 

Subjects  individually  complete  Desert  Scenario 
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Assistant  says:  “We  will  now  move  to  the  Task  room  to  continue  the  task.  Remember  the  goal  to 
participate  equally  while  maximizing  your  number  of  comments.  Please  take  your  desert 
scenario  and  questionnaire  with  you  and  place  it  in  your  manila  folder.  ” 

Researchers  move  subjects  to  Task  Room  (GSS  Room) 
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GSS  Room:  Experiment  Two 

Desert  Scenario  Script 

Facilitator  says:  “You  will  have  15  minutes  to  discuss  the  scenario  as  a  group.  The  scenario  you 
will  be  discussing  is  the  same  one  you  did  previously  as  individuals.  At  the  end  of  the  15  minutes 
you  will  each  rank  order  the  list  individually.  The  ranking  results  will  be  consolidated  as  a 
group  to  indicate  how  well  the  group  reached  consensus.  At  this  time  the  group  will  have 
another  5  minutes  to  determine  if  you  have  reached  consensus.  If  the  group  did  not  reach 
consensus  you  should  try  to  resolve  any  differences.  The  group  will  then  individually  rank  the 
items  again.  ” 

Facilitator  says:  “During  the  15-minute  discussion  period,  focus  on  discussing  each  item’s 
merits,  not  on  where  each  item  should  be  ranked.  You  will  be  able  to  rank  order  the  list  at  the 
end  of  the  15  minutes.  It  is  important  to  focus  on  discussion  not  on  how  to  rank  order  since  this 
could  shut  down  conversation.  It  is  normal  to  experience  a  lull  during  conversation,  but  this  is 
part  of  group  dynamics.  ” 

Facilitator  says:  “Researchers  will  not  answer  questions  dealing  with  the  scenario  during  this 
session,  but  will  assist  with  GSS  questions.  We  will  let  you  know  when  there  are  5  and  2  minutes 
left  in  the  session.  ’’ 

Facilitators  start  participants  in  Desert  (Categorizer). 

Facilitator  says:  “At  this  point,  you  should  have  been  invited  to  join  an  activity.  Please  click  on 

Yes.  ”  (Note  to  Researcher:  In  some  cases  this  field  will  be  blank.) 

Facilitator  says:  “Now  you  should  be  prompted  to  enter  an  author  tag.  Please  click  on  OK.  ” 

Facilitator  says:  “You  have  a  list  of  15  items  and  a  category  called  “Group  Discussion”.  Double 
Click  on  Group  Discussion.” 

Facilitator  says:  “Are  there  any  questions?” 

Facilitator  says:  “You  may  begin  group  discussion  on  the  Desert  Scenario.  ” 


10:00 

FACILITATOR  INPUT:  YOU  HAVE  5  MINUTES  LEFT  TO  DISCUSS 
THE  SCENARIO  IF  NEEDED. 

13:00 

FACILITATOR  INPUT:  YOU  HAVE  2  MINUTES  LEFT  TO  DISCUSS 
THE  SCENARIO  IF  NEEDED. 

Subjects  finish  initial  discussion. 

Facilitator  says:  “Please  close  your  Group  Discussion  Window.  ” 
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Facilitator  says:  “We  will  now  open  a  voting  tool  for  your  use.  ” 

Facilitator  starts  Desert  (Vote). 

Facilitator  says:  "You  have  been  invited  to  join  the  Voting  activity.  Please  click  yes.  Now  you 
should  see  the  original  list  of  items.  Please  begin  voting  by  re-ordering  the  list  now.  ” 

PAUSE  (1  minute) 

Facilitator  says:  "Once  you  are  satisfied  with  your  list  order,  cast  your  ballot  by  clicking  on  the 
"cast  ballot”  icon,  which  is  the  2'"^  from  the  left.  You  will  receive  a  dialogue  box  asking  you  to 
confirm  your  ballot.  Please  click  yes  and  wait  for  further  instructions.  ” 


Ensure  voting  graph  tool  is  set  as  follows: 


Facilitator:  Open  the  result  window  to  monitor  individual  votes.  When  n=number  of 
participants,  the  group  is  done.  Display  the  results  with  the  projector  and  explain  the  level  of 
group  consensus. 

Facilitator  flips  UP  projector 
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Facilitator  says:  “You  have  the  next  five  minutes  to  discuss  the  results  further  using  the  GSS.  If 
you  are  satisfied  with  the  results  let  the  facilitator  know.  If  not  you  will  be  given  the  chance  to 
vote  again  at  the  end  of  the  five  minutes.  ” 

If  group  is  satisfied  with  original  outcome  then  go  to  STEP  2  else  do  the  following: 

Facilitator  says:  “Go  ahead  and  vote  again.  ” 

Facilitator:  Open  the  result  window  to  monitor  individual  votes.  Assistant  lets  Facilitator  know 
when  the  group  is  done.  Display  the  results  with  the  projector  and  explain  the  level  of  group 
consensus. 

Facilitator  says:  “These  are  your  final  results.  “ 

Researcher  stops  Desert  (Vote). 

Facilitator  flips  DOWN  projector 

Facilitator  says:  “Lets  go  back  to  the  prep  room  to  finish  up.  ” 


Prep  Room:  Wrap-Up 

Assistant  provides  subjects  with  Post-Session  Questionnaire 
Subjects  complete  Post-Session  Questionnaire 
Assistant  debriefs  subjects 

“The  experiment  you  just  participated  in  was  designed  to  measure  the  effect  of  feedback  and 
goal  setting  on  group  performance,  compare  different  levels  of  anonymity  in  a  meeting  on  group 
performance,  study  ideation  over  time,  and  evaluate  the  influence  of  personality  types  on 
groups.  ” 

“The  experiment  collected  data  on  the  quantity  of  comments  provided,  the  quality  of  group 
decision,  the  timing  of  ideas  generated,  and  participation  rates  from  various  personality 
groups.  ” 

“I  would  like  to  thank  you  for  your  participation  in  this  experiment.  Do  you  have  any  other 
questions  about  the  experiment  you  participated  in  today  or  on  Group  Support  Systems?” 

[Pause  for  questions.] 

“Please,  if  you  know  others  who  are  likely  to  participate  in  this  experiment,  please  keep  the 
details  of  the  experiment  to  yourself  in  order  to  avoid  biasing  our  final  results  and  jeopardizing 
the  continuation  of  this  study.  ” 

Researchers  collect  all  handouts,  data,  disks,  etc.  and  ensures  all  are  labeled 
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Appendix  E:  Face-to-face  Experiment  Script 


Face-to-Face:  Pre-Experiment  Steps 

1.  Ensure  following  items  are  available: 

Big  folder  labeled  consent  forms 
4  Manila  folders 
Attached  via  paper  clip  are 
Consent  form 

Demographic/Personality  Questionnaire 

1  copy  of  Moon  Scenario 

2  Comment  Tally  Sheets  (includes  Moon  and  Desert  Scenario) 

4  copies  of  Training  Scenario 

12  Paper  copies  of  both  scenarios  (Moon  and  Desert) 

8  Pencils 

Flip  Chart  or  Whiteboard  or  Chalk  Board  with  appropriate  marking  device 
Check  out  Projector,  Laptop  and  Printer  with  paper 
Room  configuration  standardized 

Prep  Room:  Introduction 

1.  When  subjects  arrive,  introduce  yourself.  Have  subjects  wait  in  the  prep  room.  Tell  subjects 
“The  task  will  begin  when  all  participants  have  arrived.  ” 

2.  Once  all  participants  have  arrived,  have  all  subjects  sit  down  in  the  prep  room. 

3.  Facilitator  says:  “Welcome  to  the  study.  I’m  XX  and  this  is  XX.  We  are  AFIT  students 
conducting  an  experiment  for  our  Masters  degree.  We  will  be  asking  you  some  questions  about 
yourself.  Our  study  looks  at  how  different  types  of  groups  interact  to  solve  a  problem.  During 
the  course  of  this  experiment  you  will  be  asked  to  complete  three  questionnaires,  receive  some 
group  interaction  training,  and  conduct  tasks  individually  and  as  a  group.  About  halfway 
through  this  two  hour  experiment  you  will  be  given  a  short  break.  ” 

4.  Facilitator  says:  “My  assistant  will  now  hand  out  a  manila  folder  with  some  attached 
information.  Please  don’t  look  at  the  attached  information  until  asked.  ” 

5.  Assistant  provides  participants  with.manila  folder. 

6.  Facilitator  says:  “To  begin,  please  remove  the  consent  form  from  the  manila  folder.  This 
form  indicates  your  rights  as  a  participant  in  the  study.  Please  read  the  consent  form  and  print 
and  sign  your  name  at  the  bottom  of  the  page.  Your  participation  is  voluntary.  If  at  any  time  you 
want  to  stop  please  let  the  facilitator  know.  ” 
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7.  Subjects  read  and  sign  (if  applicable)  Consent  Form 

8.  Assistant  collects  consent  forms 

9.  Facilitator  says,  “This  is  the  only  place  your  name  will  be  recorded  during  this  experiment.  ” 

10.  Assistant  puts  consent  forms  in  big  folder  labeled  consent  forms. 

11.  Facilitator  says:  “We  would  now  like  you  to  fill  out  the  individual  characteristics 
questionnaire  attached  to  the  manila  folder.  All  responses  to  this  questionnaire  are  completely 
confidential  and  will  not  be  associated  with  you  as  an  individual.  Use  the  rating  scale  provided 
to  indicate  how  accurately  each  statement  describes  you.  Think  about  yourself  as  you  generally 
are  now  and  not  as  you  wish  to  be  in  the  future.  Please  read  each  statement  carefully.  Does 
anyone  have  any  questions?’* 

12.  Subjects  complete  questionnaire. 

13.  Facilitator  says:  “Please  place  the  questionnaire  in  the  manila  folder.  ’’ 

14.  Facilitator  says:  “Now  lets  complete  a  problem  solving  task  individually.  Please  read  the 
scenario  and  complete  the  exercise.  It  will  take  you  approximately  5  minutes  to  complete  the 
exercise.  If  you  finish  early,  please  remain  quiet  until  everyone  completes  the  exercise.  Please 
remove  the  scenario  from  your  manila  folder.  ” 

15.  Facilitator  says:  “Please  begin.” 

16.  Subjects  individually  complete  Moon  Scenario 

17.  Facilitator  says:  “Please  place  the  Moon  Scenario  in  the  manila  folder.  ” 

18.  Facilitator  says:  “Before  we  move  to  another  room  let’s  discuss  group  decision  making  and 
problem  solving  in  general .  The  first  step  is  for  the  group  to  discuss  the  problem  and  all 
pertinent  issues  related  to  the  problem.  One  method  often  used  to  do  this  is  “brainstorming” 
during  which  ideas  are  freely  generated  and  not  judged  on  quality  or  feasibility.  Once  the 
brainstorming  session  is  complete,  the  group  then  attempts  to  reach  consensus  on  a  solution. 
This  does  not  necessarily  mean  all  individuals  completely  agree  with  the  groups’  decision,  but 
the  decision  is  one  that  all  can  endorse.  There  are  dijferent  methods  groups  use  to  reach 
consensus,  one  of  which  is  voting.  If  the  results  of  the  group  vote  indicate  agreement,  then 
consensus  is  reached.  If  the  group  does  not  have  agreement,  further  discussion  may  be  required 
to  reach  consensus.  Remember  the  purpose  of  this  study  is  to  look  at  how  different  types  of 
groups  interact  to  solve  a  problem.  Does  anyone  have  any  questions  before  we  move  to  the  next 
phase  of  the  study.  ” 

19.  Facilitator  says:  “Please  pick  up  your  manila  folder  and  follow  me.  ” 

Assistant  moves  subjects  to  Task  Room  (Face-to-Face  Room) 
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Face-to-Face  Room:  Training 

Training  Script 

Facilitator  says:  “Before  we  begin  the  actual  group  problem-solving  scenario,  we  will  first  guide 
you  through  a  brief  training  session.  Let’s  begin.  ” 

Assistants  hand  out  training  scenario. 

Facilitator  says:  “Your  group’s  task  is  to  come  to  consensus  on  how  to  rank  order  the  list  of 
names.  Your  goal  is  to  discuss  possible  ways  your  group  could  rank  order  the  names.  Any  and 
all  comments  are  valuable,  including  ideas  on  how  to  rank  the  names,  and  your 
thoughts/opinions  of  each  other’s  ideas.  It  will  take  about  5  minutes  to  discuss  the  task.  At  the 
end  of  the  session  you  will  each  individually  rank  order  the  list  based  on  the  groups  chosen 
method.  Individual  results  will  be  combined  to  determine  group  consensus.  Any  questions?” 

Facilitator  says:  “Don’t  worry  if  you  see  the  facilitators  taking  notes  during  the  study  we  are 
writing  notes  to  on  how  your  group  interacts.  Please  begin  discussing  the  task.  ” 

Assistant:  Time  hack  for  5  minutes. 

Assistant  notifies  facilitator  when  time  reaches  5  minutes. 

Facilitator  says:  “Please  stop  discussing  the  task  at  this  time.  ” 

Facilitator  says:  “You’ve  had  enough  time  to  discuss  possible  ways  to  rank  order  the  list  of 
names.  Now  it’s  time  to  actually  rank  the  names.  Hopefully,  during  the  discussion  period,  your 
group  was  able  to  come  to  consensus  on  how  to  rank  the  list.  ” 

Facilitator  says:  “Go  ahead  and  rank  the  list  of  names  individually  using  the  method  chosen  by 
the  group.  When  done  hand  your  list  to  the  facilitator.  ” 

Assistant  collects  training  scenarios. 

Facilitator  reviews  list  with  group  discussing  group  consensus. 

Facilitator  says:  “During  the  actual  problem-solving  tasks  following  this  training,  your  group 
will  be  given  5  minutes  after  the  initial  vote  to  allow  your  group  to  determine  if  everyone  is 
satisfied  with  the  final  solution,  or  if  further  discussion  is  needed.  ”■ 

Training  exercise  complete. 

Facilitator  says:  “We  will  now  start  the  first  exercise.  Please  do  not  discuss  the  scenario  until 
instructed.  ” 
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Face-to-Face  Room:  Experiment  One 


Moon  Scenario  Script 

Facilitator  says:  “You  will  have  15  minutes  to  discuss  the  scenario  as  a  group.  The  scenario  you 
will  be  discussing  is  the  same  one  you  did  previously  as  individuals.  At  the  end  of  the  15  minutes 
you  will  each  rank  order  the  list  individually.  The  ranking  results  will  be  consolidated  as  a 
group  to  indicate  how  well  the  group  reached  consensus.  At  this  time  the  group  will  have 
another  5  minutes  to  determine  if  they  have  reached  consensus.  If  your  group  does  not  reach 
consensus,  you  should  try  to  resolve  any  differences.  The  group  will  then  individually  rank  the 
items  again.  ” 

Facilitator  says:  “During  the  15-minute  discussion  period,  focus  on  discussing  each  item ’s 
merits,  not  on  where  each  item  should  be  ranked.  You  will  be  able  to  rank  order  the  list  at  the 
end  of  the  15  minutes.  It  is  important  to  focus  on  discussion  not  on  how  to  rank  order  since  this 
could  shut  down  conversation.  It  is  normal  to  experience  a  lull  during  conversation,  but  this  is 
part  of  group  dynamics.  ” 

Facilitator  says:  “Researchers  will  not  answer  questions  dealing  with  the  scenario  during  this 
session.  We  will  let  you  know  when  there  are  5  and  2  minutes  left  in  the  session.  ” 


Facilitator  says:  “Are  there  any  questions?” 


Facilitator  says:  “You  may  begin  Group  Discussion  on  the  Moon  Scenario.  ” 


Facilitator  and  Assistant  keep  track  of  comments  on  comment  tally  sheet. 


10:00  YOU  HAVE  5  MINUTES  LEFT  TO  DISCUSS  THE  SCENARIO  IF 

NEEDED. 


1 3 :00  YOU  HAVE  2  MINUTES  LEFT  TO  DISCUSS  THE  SCENARIO  IF 

NEEDED. 


Subjects  finish  initial  discussion. 

Facilitator  says:  “Now  lets  vote.” 

Assistant  hands  out  another  copy  of  Moon  Scenario. 

Facilitator  says:  “Please  fill  the  scenario  out  individually.  When  finished  give  your  copy  to  the 
facilitator.  ” 

Subjects  individually  rank  Moon  Scenario  items  and  give  to  assistant. 

When  the  group  is  done: 
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Assistant  calculates  means  on  laptop  in  Excel,  sorts  the  list  by  mean,  and  projects  the  mean  and 
each  subject’s  rankings  on  the  wall. 

Facilitator  explains  the  level  of  group  consensus. 

Facilitator  says:  “You  have  the  next  five  minutes  to  discuss  the  results  further.  If  you  are 
satisfied  with  the  results  let  the  facilitator  know.  If  not  you  will  be  given  the  chance  to  vote  again 
at  the  end  of  the  five  minutes.” 

If  group  is  satisfied  with  original  outcome  then  go  to  STEP  2  else  do  following: 

Assistant  hands  out  another  copy  of  Moon  Scenario. 

Facilitator  says:  “Go  ahead  and  vote  again.  ” 

When  the  group  is  done: 

Assistant  calculates  means  on  laptop  in  Excel,  sorts  the  list  by  mean,  and  projects  the  mean  and 
each  subject’s  rankings  on  the  wall. 

Facilitator  explains  the  level  of  group  consensus. 

Facilitator  says:  “These  are  your  final  results.  “ 

Facilitator  says:  “Feel  free  to  take  a  quick  5-minute  break  in  the  prep  room.  ” 

Assistant  counts  number  of  conunents  per  subject  comment  tally  sheet  and  create  appropriate 
feedback  and  goal  charts. 

Assistant  prints  the  spreadsheet. 
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Prep  Room:  Feedback 

Subjects  come  back  from  break 

Assistant  says:  “Wie  would  now  like  to  give  you  a  short  questionnaire  concerning  your  groups 
ranking  on  the  task  you  just  completed.  ” 

Assistant  provides  subjects  with  5  item  commitment  to  ranking  Questionnaire 
Goal  and  No  Feedback  or  Goal  Feedback  Script 


Assistant  says:  ‘‘During  the  group  exercise  just  completed,  your  group  worked  together  to 
solve  a  problem.  Studies  have  shown  that  when  individual  members  of  the  group  participate 
fairly  equally,  the  meeting  will  produce  better  results.  For  example,  as  you  can  see  in  the 
graph  (show  graph  of  equal  proportion)  the  participation  rates  were  almost  equal  among  the 
group  participants.  The  next  graph  shows  participation  rates  where  participants  did  not 
participate  equally.  What  problem  can  result  from  the  unequal  levels  of participation?’’ 

(Wait  for  group  to  respond. . .  Look  for  an  answer  such  as  subject  4  did  not  participate  as 
much  and  he  may  have  had  the  best  answer  while  subject  2  dominated  the  meeting  with  his 
ideas.  If  group  does  not  submit  the  answer  looked  for,  provide  an  explanation. 

Assistant  says:  “Studies  have  also  shown  that  the  more  comments  input  during  a  meeting,  the 
greater  the  chances  to  reach  a  high  quality  decision.  In  other  words,  the  more  ideas  that  are 
generated  the  better  the  chance  the  optimum  solution  will  be  found  in  those  comments. " 

Assistant  says:  GOAL  STATEMENT:  “In  the  next  task,  try  to  participate  equally  while 
maximizing  your  number  of  comments.  ” 

IF  GIVING  FEEDBACK  READ  THE  NEXT  SECTION,  IF  NOT  STOP  AND  PROCEED  TO 
NEXT  TASK 

Assistant  says:  “I  will  now  show  each  of  you  a  graph  showing  your  participation  level  in  the 
previous  task.  “ 

Assistant  provides  an  explanation  of  the  feedback. 

Assistant  says:  “Next  to  the  bar  graph  of  each  individual  show  their  score.  Once  all  subjects 
have  looked  at  the  paper  give  them  the  paper  to  start  the  desert  scenario.  ” 

Assistant  says:  “You  will  now  be  given  5  minutes  to  complete  the  desert  scenario.  Please  follow 
the  directions  on  the  page.  ” 

Subjects  individually  complete  Desert  Scenario 
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Assistant  says:  “We  will  now  move  to  the  Task  room  to  continue  the  task.  Remember  the  goal  to 
participate  equally  while  maximizing  your  number  of  comments.  Please  take  your  desert 
scenario  and  questionnaire  with  you  and  place  it  in  your  manila  folder.  ” 

15.  Researchers  move  subjects  to  Task  Room  (Non-GSS  Room) 


Face-to-Face  Room:  Experiment  Two 

Desert  Scenario  Script 

Facilitator  says:  “You  will  have  15  minutes  to  discuss  the  scenario  as  a  group.  The  scenario  you 
will  be  discussing  is  the  same  one  you  did  previously  as  individuals.  At  the  end  of  the  15  minutes 
you  will  each  rank  order  the  list  individually.  The  ranking  results  will  be  consolidated  as  a 
group  to  indicate  how  well  the  group  reached  consensus.  At  this  time  the  group  will  have 
another  5  minutes  to  determine  if  they  have  reached  consensus.  If  your  group  does  not  reach 
consensus,  you  should  try  to  resolve  any  differences.  The  group  will  then  individually  rank  the 
items  again.  ” 

Facilitator  says:  “During  the  15-minute  discussion  period,  focus  on  discussing  each  item’s 
merits,  not  on  where  each  item  should  be  ranked.  You  will  be  able  to  rank  order  the  list  at  the 
end  of  the  15  minutes.  It  is  important  to  focus  on  discussion  not  on  how  to  rank  order  since  this 
could  shut  down  conversation.  It  is  normal  to  experience  a  lull  during  conversation,  hut  this  is 
part  of  group  dynamics.  ” 

Facilitator  says:  “Researchers  will  not  answer  questions  dealing  with  the  scenario  during  this 
session.  We  will  let  you  know  when  there  are  5  and  2  minutes  left  in  the  session.  ” 

Facilitator  says:  “Are  there  any  questions?” 

Facilitator  says:  “You  may  begin  Group  Discussion.  ” 


Facilitator  and  Assistant  keep  track  of  comments  on  comment  tally  sheet. 


10:00 

YOU  HAVE  5  MINUTES  LEFT  TO  DISCUSS  THE  SCENARIO  IF 
NEEDED. 

13:00 

YOU  HAVE  2  MINUTES  LEFT  TO  DISCUSS  THE  SCENARIO  IF 
NEEDED. 

Subjects  finish  initial  discussion. 
Facilitator  says:  “Now  lets  vote.” 
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Assistant  hands  out  another  copy  of  Desert  Scenario. 


Facilitator  says:  “Please  fill  the  scenario  out  individually.  When  finished  give  your  copy  to  the 
facilitator.  ” 

Subjects  individually  rank  Desert  Scenario  items  and  give  to  assistant. 

When  the  group  is  done: 

Assistant  calculates  means  on  laptop  in  Excel,  sorts  the  list  by  mean,  and  projects  the  mean  and 
each  subject’s  rankings  on  the  wall. 

Facilitator  explains  the  level  of  group  consensus. 

Facilitator  says:  “You  have  the  next  five  minutes  to  discuss  the  results  further .  If  you  are 
satisfied  with  the  results  let  the  facilitator  know.  If  not  you  will  be  given  the  chance  to  vote  again 
at  the  end  of  the  five  minutes.  ” 

If  group  is  satisfied  with  original  outcome  then  go  to  STEP  2  else  do  following: 

Assistant  hands  out  another  copy  of  Desert  Scenario. 

Facilitator  says:  “Go  ahead  and  vote  again.  ” 

When  the  group  is  done: 

Assistant  calculates  means  on  laptop  in  Excel,  sorts  the  hst  by  mean,  and  projects  the  mean  and 
each  subject’s  rankings  on  the  wall. 

Facilitator  explains  the  level  of  group  consensus. 

Facilitator  says:  “These  are  your  final  results. 

Assistant  counts  number  of  comments  per  subject  comment  tally  sheet. 

Prep  Room:  Wrap-Up 

Assistant  provides  subjects  with  Post-Session  Questionnaire 
Subjects  complete  Post-Session  Questionnaire 
Assistant  debriefs  subjects 

“The  experiment  you  fust  participated  in  was  designed  to  measure  the  effect  of  feedback  and 
goal  setting  on  group  performance,  compare  different  levels  of  anonymity  in  a  meeting  on  group 
performance,  study  ideation  over  time,  and  evaluate  the  influence  of  personality  types  on 
groups.  ” 

“The  experiment  collected  data  on  the  quantity  of  comments  provided,  the  quality  of  group 
decision,  the  timing  of  ideas  generated,  and  participation  rates  from  various  personality 
groups.  ” 

“I  would  like  to  thank  you  for  your  participation  in  this  experiment.  Do  you  have  any  other 
questions  about  the  experiment  you  participated  in  today  or  on  Group  Support  Systems?” 
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[Pause  for  questions.] 

“Please,  if  you  know  others  who  are  likely  to  participate  in  this  experiment,  please  keep  the 
details  of  the  experiment  to  yourself  in  order  to  avoid  biasing  our  final  results  and  jeopardizing 
the  continuation  of  this  study.  ” 

Researchers  collect  all  handouts,  data,  etc.  and  ensures  all  are  labeled 
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Appendix  F:  Consent  Form 


Study  Overview 

Welcome  to  the  experiment.  The  following  is  a  general  description  of  the  study  and  a 
reminder  of  your  rights  as  a  potential  subject.  As  in  any  study,  your  participation  is  completely 
voluntary.  If  now,  or  at  any  point  during  the  study,  you  decide  that  you  do  not  want  to  continue 
participating,  please  let  the  experimenter  know  and  you  will  be  dismissed  without  penalty.  Also, 
please  remember  that  your  name  will  not  be  associated  with  any  of  the  information  that  you 
provide  during  the  study.  All  of  the  information  you  provide  is  absolutely  anonymous  and 
confidential. 

In  this  study,  you  will  be  working  as  part  of  a  group  to  complete  two  group  tasks.  You  will 
also  be  asked  to  complete  two  questionnaires  during  the  study.  You  will  first  be  given  a 
questionnaire  to  complete,  then  you  will  complete  the  first  task  as  a  group,  after  a  short  break 
you  will  be  given  the  second  task  to  complete  as  a  group,  and  finally,  you  will  be  given  a  second 
questionnaire  to  complete.  The  experimenter  will  give  you  more  specific  instructions  later  in  the 
study.  If  you  have  any  questions  or  concerns  at  this  time,  please  inform  the  experimenter. 

For  further  information 


The  Air  Force  Institute  of  Technology  faculty  members  responsible  for  conducting  this 
research  are  Maj.  Michael  Morris  and  Maj.  Paul  Thurston.  They  would  be  happy  to  address  any 
of  your  questions  or  concerns  regarding  this  study.  Maj.  Morris  can  be  reached  at  255-3636  ext 
4578  and  Maj.  Thurston  can  be  reached  at  255-6565  ext  4315. 

If  you  would  like  to  participate  in  this  study,  please  sign  in  the  space  provided.  Your 
signature  indicates  that  you  are  aware  of  each  of  the  following:  1)  the  general  procedure  to  be 
used  in  this  study,  2)  your  right  to  discontinue  participation  at  any  time,  and  3)  you  and  your 
name  will  not  be  associated  with  any  of  the  information  you  provide. 


Printed  Name: 


Signature: 


Date: 
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Appendix  G:  Personality  Questionnaire 


Answer  the  questions  using  the  following  scale. 

1.  Very  Inaccurate 

2.  Moderately  Inaccurate 

3.  Neither  Accurate  Nor  Inaccurate 

4.  Moderately  Accurate 

5.  Very  Accurate 

_ 1. 1  trust  others. 

_ 2. 1  complete  tasks  successfully. 

_ 3. 1  would  never  cheat  on  my  taxes. 

_ 4. 1  like  order. 

_ 5. 1  am  easy  to  satisfy. 

_ 6. 1  avoid  mistakes. 

_ 1 . 1  believe  that  people  are  essentially  evil. 

_ 8. 1  don’t  see  the  consequences  of  things. 

_ 9. 1  obstruct  others’  plans. 

_ 10. 1  am  not  bothered  by  disorder. 

_ 11. 1  hold  a  grudge. 

_ 12. 1  often  make  last-minute  plans. 

_ 13. 1  believe  that  others  have  good  intentions. 

_ 14. 1  excel  in  what  I  do. 

_ 15. 1  stick  to  the  rules. 

_ 16. 1  like  to  tidy  up. 

_ 17. 1  can’t  stand  confrontations. 

_ 18. 1  choose  my  words  with  care. 
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1.  Very  Inaccurate 

2.  Moderately  Inaccurate 

3.  Neither  Accurate  Nor  Inaccurate 

4.  Moderately  Accurate 

5.  Very  Accurate 

_ 19. 1  am  aware  of  others. 

_ 20. 1  have  little  to  contribute. 

_ 21. 1  take  advantage  of  others. 

_ 22. 1  am  not  bothered  by  messy  people. 

_ 23. 1  get  back  at  others. 

_ 24. 1  act  without  thinking. 

_ 25. 1  trust  what  people  say. 

_ ^26. 1  handle  tasks  smoothly. 

_ 27. 1  use  flattery  to  get  ahead. 

_ 28. 1  want  everything  to  be  “just  right”. 

_ 29. 1  hate  to  seem  pushy. 

_ 30. 1  stick  to  my  chosen  path. 

_ 31. 1  suspect  hidden  motives  in  others. 

_ 32. 1  don’t  understand  things. 

_ 33. 1  pretend  to  be  concerned  for  others. 

_ 34. 1  leave  my  belongings  around. 

_ 35. 1  insult  people. 

_ 36. 1  do  crazy  things. 

_ 37. 1  believe  that  people  are  basically  moral. 

_ 38. 1  am  sure  of  my  ground. 
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1.  Very  Inaccurate 

2.  Moderately  Inaccurate 

3.  Neither  Accurate  Nor  Inaccurate 

4.  Moderately  Accurate 

5.  Very  Accurate 

_ 39. 1  use  others  for  my  own  ends. 

_ ^40. 1  love  order  and  regularity. 

_ ^41. 1  have  a  sharp  tongue. 

_ ^42. 1  jump  into  things  without  thinking. 

_ ^43. 1  distrust  people. 

_ ^44. 1  misjudge  situations. 

_ ^45. 1  put  people  under  pressure. 

_ 46. 1  do  things  according  to  a  plan. 

_ ^47. 1  yell  at  people. 

_ ^48. 1  rush  into  things. 

_ ^49. 1  believe  in  human  goodness. 

_ 50. 1  come  up  with  good  solutions. 

_ 51. 1  know  how  to  get  around  the  rules. 

_ 52. 1  leave  a  mess  in  my  room. 

_ 53. 1  contradict  others. 

_ 54. 1  make  rash  decisions. 

_ 55. 1  think  that  all  will  be  well. 

_ 56. 1  know  how  to  get  things  done. 

_ 57. 1  cheat  to  get  ahead. 

_ 58. 1  often  forget  to  put  things  back  in  their  proper  place. 
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59. 1  love  a  good  fight. 

60. 1  like  to  act  on  a  whim. 


_ Male _ Female  _ Married _ Single  Age; _ 

Highest  Education  Level  Completed  (please  choose  one): 

_ High  School  _ _  Bachelor’ s  Degree  _ Graduate  Degree 

_ Some  College  _ Some  Graduate  Studies  _ Post  Graduate  Degree 

For  Bachelor’s,  Graduate,  and  Post  Graduate  Degree  recipients,  please  enter  the  type  of  degree 
conferred  (e.g.,  BS  Computer  Science,  MBA,  BA  MIS,  etc.): 

Bachelor’ s: _ _ 

Graduate: _ _ _ 

Post  Graduate: _ 

If  active  duty  military,  enter  the  number  of  years  you’ve  spent  on  active  duty; _ 

If  civilian  with  prior  military  service,  enter  the  number  of  years  spent  on  active  duty: _ , 

and  the  number  of  years  of  paid  employment  not  including  prior  military  service: _ 

If  civilian  with  no  prior  military  service,  enter  the  number  of  years  of  paid  emplo5maent: _ 

Current  occupational  specialty  or  occupation: _ 

(e.g.,  Communications  &  Information,  Logistics,  Management,  Teacher,  etc.) 

Number  of  years  supervisory  experience: _ 

Approximately  how  many  years  have  you  used  a  computer? 

Less  than  1  1-5  _ 6-10  _ 10  or  more 


Approximately  how  many  hours  per  week  do  you  currently  use  a  computer  (work  and  home)? 
_ 0-10  _ 11-20  _ 21-30  _ 31  or  more 


Answer  the  remaining  questions  using  the  following  scale. 

1  -  Very  Inaccurate 

2  -  Moderately  Inaccurate 

3  -  Neither  Accurate  Nor  Inaccurate 

4  -  Moderately  Accurate 

5  -  Very  Accurate 

_ I  feel  comfortable  using  e-mail 

_ I  feel  comfortable  programming  a  computer 

_ I  feel  comfortable  using  MS  Word  and  other  desktop  software  tools 

_ I  am  a  proficient  typist 

_ I  feel  comfortable  navigating  around  the  Internet 

_ I  am  knowledgeable  about  computer  networks 

_ I  am  comfortable  learning  how  to  use  new  computer  software 

_ Overall,  I  am  proficient  at  using  personal  computers  (PCs) 
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Appendix  H:  Moon  Scenario 

You  are  a  member  of  a  space  crew  originally  scheduled  to  rendezvous  with  a  mother  ship 
on  the  lighted  surface  of  the  moon.  Due  to  mechanical  difficulties,  however,  your  ship  was 
forced  to  land  at  a  spot  some  200  miles  from  the  rendezvous  point.  During  re-entry  and  landing, 
much  of  the  equipment  aboard  was  damaged  and,  since  survival  depends  on  reaching  the  mother 
ship,  the  most  critical  items  available  must  be  chosen  for  the  200-mile  trip. 

The  15  items  left  intact  and  undamaged  after  landing  are  listed  below.  Your  task  is  to 
rank  them  in  terms  of  their  necessity  to  your  crew  in  reaching  the  rendezvous  point.  Place  the 
number  1  by  the  most  crucial  item,  the  number  2  by  the  second  most  crucial,  and  so  on  through 
number  15,  the  least  important. 

_ Box  of  matches 

_ First-aid  kit  containing  injection  needles 

_  Five  gallons  water 

_  Food  concentrate 

_  Life  raft 

_ Magnetic  compass 

_  One  case  dehydrated  milk 

_  Parachute  silk 

_  Portable  heating  unit 

_  Signal  flares 

_  Solar-powered  FM  receiver  transmitter 

_ Stellar  map  (of  the  moon's  constellation) 

_ Two  .45-caliber  pistols 

_  Two  100-pound  tanks  of  oxygen 

_  50  ft.  of  nylon  rope 
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Appendix  I:  Commitment  to  Group  Ranking  Questionnaire 


Answer  the  questions  using  the  following  scale 


1.  Strongly  Disagree 

2.  Disagree 

3.  Disagree  Somewhat 

4.  Neither  Agree  Nor  Disagree 

5.  Agree  Somewhat 

6.  Agree 

7.  Strongly  Agree 

_ I  believe  my  group’s  solution  is  better  than  the  ranking  I  developed  myself. 

_  I  would  be  willing  to  make  other  decisions  based  on  my  group’s  ranking  because  it  is  so 

accurate. 

_  I  would  be  willing  to  argue  my  group’s  solution  to  another  group. 

_  I  understand  the  reasons  why  we  ranked  items  in  their  particular  order. 

_  I  agree  with  the  reasons  why  we  ranked  items  in  their  particular  order. 
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Appendix  J:  Desert  Scenario 


It  is  approximately  10:00  AM  in  mid  August  and  you  have  just  crash-landed  in  the 

Sonora  Desert  in  southwestern  United  States.  The  twin  engine  plane,  containing  the  bodies  of 

the  pilot  and  the  co-pilot,  has  completely  burned.  Only  the  airframe  remains.  None  of  the  rest  of 

you  have  been  injured.  The  pilot  was  unable  to  notify  anyone  of  your  position  before  the  crash. 

However,  he  had  indicated  before  impact  that  you  were  70  miles  south  -  southwest  from  a 

mining  camp  which  is  the  nearest  known  habitation  and  that  you  were  approximately  65  miles 

off  the  course  that  was  filed  in  your  Flight  Plan. 

Before  the  plane  caught  fire  your  Patrol  was  able  to  salvage  the  15  items  listed  on  the 

attached  sheet.  Your  task  is  to  rank  these  items  according  to  their  importance  to  your  survival. 

Place  the  number  1  by  the  most  cmcial  item,  the  number  2  by  the  second  most  crucial,  and  so  on 

through  number  15,  the  least  important. 

_  A  pair  of  sunglasses  per  person 

_  Book  entitled  “Edible  Animals  of  the  Desert” 

_ Bottle  of  salt  tablets  (1000  tablets) 

_  Compress  kit  and  gauze 

_  Cosmetic  Mirror 

_  Flashlight 

_ Magnetic  compass 

_  One  liter  of  water  per  person 

_  One  top  coat  per  person 

_  Parachute  (red  and  white) 

_  Penknife 

_ Plastic  Raincoat  (large  size) 

_  Sectional  Air  Map  of  the  Area 

_  2  liters  of  100%  proof  vodka 

_  .45  caliber  pistol 
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Appendix  K:  Box  and  Whiskers  Defined 


Box  plots 

Summary  plot  based  on  the  median,  quartiles,  and  extreme  values.  The  box  represents  the 
interquartile  range,  which  contains  the  50%  of  values.  The  whiskers  are  lines  that  extend  from 
the  box  to  the  highest  and  lowest  values,  excluding  outliers  and  extreme  points.  A  line  across  the 
box  indicates  the  median.  (SPSS  10.0.7  documentation,  2000) 

Outliers  -  Cases  where  a  value  is  between  1.5  and  3  box  lengths  from  the  upper  or  lower  edge  of 
the  box.  These  points  are  signified  on  the  boxplot  with  an  “O”  and  the  session  number. 

Extreme  Points  —  Cases  where  a  value  is  greater  than  3  box  lengths  from  the  upper  or  lower 
edge  of  the  box.  These  points  are  signified  on  the  boxplot  with  an  and  the  session  number. 

NOTE:  Both  Outliers  and  Extreme  point  values  are  used  in  calculating  the  mean. 
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and  process  anonymity  influence  participation  rates,  quality  of  the  group  decision,  consensus  levels,  user  attitudes,  and  user  satisfaction  with  the  group  outcome. 
Groups  participated  in  conditions  of  total  anonymity,  process  anonymity,  and  no  anonymity.  The  no  anonymity  condition  included  face-to-face  discussion  and  use  of 
a  GSS  in  which  comments  were  labeled  and  stations  were  labeled  with  placards.  The  results  of  the  study  were  underwhelming.  In  general,  it  was  found  that  face-to- 
face  groups  participated  at  higher  levels,  achieved  a  higher  group  decision  quality,  reached  higher  consensus  and  were  more  satisfied  with  the  group  outcome.  These 
findings  cannot  be  attributed  to  the  lack  of  process  and  content  anonymity  however,  because  the  GSS  labeled  with  placard  treatment  did  not  achieve  the  same  results. 
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